Summary:
1) Add sync status
2) Combine sync status and phab status to use one unified conduit call,
i.e. differential.querydiffhashes
Test Plan:
cd hg-crew
make local
cd ../fb-hgext
make local
cd tests
../../hg-crew/tests/run-tests.py test-phabstatus.t
../../hg-crew/tests/run-tests.py test-syncstatus.t
Reviewers: wqfish, lcharignon, #sourcecontrol, wez, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3651915
Tasks: 10100400
Signature: t1:3651915:1470340328:bf003006f6afe9b86a40f204e150e0d12350c21d
Summary:
This is mainly to address the same issue as D3609747: prevent issues caused
by running hg as root. The difference is this extension reads owner and uid
dynamically without config when initializing of a localrepo object.
As a side effect it covers more situations like running hg in others' repos, while
being less restrictive for some other commands like `sudo hg version`.
Test Plan: Added a new test
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: ikostia, rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3619997
Tasks: 11723374
Signature: t1:3619997:1470176448:beaf53c09fca498206767641ffa4315a744ee07e
Test Plan: run tree_dump on a fastmanifest file. did some spot checks against the text manifest with the same sha (`hg debugdata -m`)
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3585843
Tasks: 12119301
Signature: t1:3585843:1469050470:70729d5601f935f2de80f89a7ad83590bcb8dd00
Summary:
Because we now prune in parallel to priming the cache, we no longer need the prune at the end. The only scenario where we still need to prune is where we never enter the priming loop, i.e., when the revset is empty.
Depends on D3545267, D3544997
Test Plan: since we don't prune in many circumstances any more, the test output is slighty affected. otherwise, the tests pass.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3547219
Signature: t1:3547219:1468373775:e4535e3a66fb362385a23b992749eb7c3fcf7ea1
Summary:
During chg pull or push over ssh, ssh is started by chgserver which does not
have a controlling tty. Therefore the ssh process won't be able to ask for
passwords interactively.
This is actually a hard issue because an unprivileged process without a ctty
cannot attach to a ctty of another process.
The discussion at upstream tends to make it clear it's part of limitations
of chg. Therefore if we decide to workaround it, it has to live outside core,
thus fb-hgext.
GUI ssh-askpass is actually a good and clean choice. See D3510178 and D3515604.
However, they are for OS X but not Linux.
This diff is a very hacky solution to make ssh-askpass works in terminal.
It starts a "tty server" providing tty I/O fds and set `SSH_ASKPASS` to use a
custom script talking to the "tty server".
Test Plan:
Run the new test. Start a sshd locally and try:
```
$ hg push ssh://root@localhost/tmp
pushing to ssh://root@localhost/tmp
root@localhost's password:
remote: Permission denied (publickey,password).
abort: no suitable response from remote hg!
$ chg push ssh://root@localhost/tmp
pushing to ssh://root@localhost/tmp
==== SSH Authenticating ====
root@localhost's password:
remote: Permission denied (publickey,password).
abort: no suitable response from remote hg!
```
Reviewers: #mercurial, ttung, mpm
Reviewed By: mpm
Subscribers: durham, mpm, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3577509
Tasks: 12029680
Signature: t1:3577509:1469467700:cd93565bd47e535bb4cb41fcdaa39e45dddfae28
Summary:
All of hg journal is now fully upstreamed to mercurial core and remotenames.
Remove the outdated copy here.
Test Plan: --
Reviewers: #mercurial, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3611123
Tasks: 10804988
Summary:
The old scm-prompt uses this longer, less awesome name.
Let's be backwards compatible with this and all that jazz.
Test Plan: tests pass under bash and zsh
Reviewers: #sourcecontrol, ttung, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3597315
Signature: t1:3597315:1469153052:98600400bb3cabf561294fd62f7cc601c280a1bd
Summary:
Previously, aliases could screw up scm-prompt. We had seen earlier
issues with this (thus the --color=never), but this is a more generic and
robust solution, applied everywhere.
We can't use full paths because they differ on different hosts, so we still
rely on a reasonable $PATH, as we always have.
Test Plan: ran scm-prompt tests under bash and zsh
Reviewers: #sourcecontrol, ttung, zamsden
Reviewed By: zamsden
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3597300
Tasks: 12298139
Signature: t1:3597300:1469060969:e88be6bd79234167f039f2964117a738ffbb95f5
Summary:
The `.py` tests are different from `.t` ones. They need special care about
`import`.
Also adds unlink `socketpath` to make sure the `.py` file runs directly
with python with additional effort to clean up the test dir.
Test Plan:
```
unset PYTHONPATH
run-tests.py test-patchpython.py
python2 test-patchpython.py
```
Reviewers: ttung, #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3579579
Signature: t1:3579579:1468848206:3711a4714080fbcc2d4360ded8316603be48fa25
Summary:
The tests that are modifying the hgrc are blacklisted for
sqldirstate because they are effectively switching exitension off.
Test Plan: tests are passing
Reviewers: #mercurial, ttung, durham, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3559268
Signature: t1:3559268:1468453765:38140f2ab4c392dc2aeefec230cbd1d4bb1b7170
Summary:
It's a common mistake that our tests require foreign extensions (namely evolve
and remotenames) without checking them first.
This diff adds checks to catch these mistakes, adds missing checks, and unifies
our checking logic using `require-ext.sh`, which is aware of `hgext3rd` and
prints skip message.
This affects `arc lint` so hopefully our new testing code would be free of this
kind of mistakes.
Test Plan: `arc lint` would catch errors
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3550977
Signature: t1:3550977:1468455857:e849dfd9e3cbc446cc6e6c662050ee88a3366e6c
Summary: `prune` is essentially `makeroomfor(0, set())`.
Test Plan: pass existing unit tests. output is slightly different, but that's just because we no longer output that debugging line.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3547069
Signature: t1:3547069:1468352305:6185132648f871e26d3cfd449c059523b5eb6d6b
Summary: It's caught by the contbuild script. I forgot to change this test.
Test Plan: Run `test-patchpython.t`.
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3550459
Signature: t1:3550459:1468349937:07123651fb103d283a382d1323d9f69a5c5d81b7
Blame Revision: D3534311
Summary:
Be a better citizen under system python path.
Fix all tests issues and change setup.py to use glob pattern to include
all extensions.
Test Plan:
Run tests and `make local`.
Also build and install the package and run `hg sl` in major repos.
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3534311
Signature: t1:3534311:1468275426:fe122646c8bd6c541e1889e73e9df28f86747ff2
Summary: Currently, this test breaks! This is awful because that means for users, when the cache becomes full, we never accept new manifests. :( :( :(
Test Plan: it breaks. :( did i say :( ?
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3545060
Tasks: 12136039
Signature: t1:3545060:1468279919:ff8928557c8ad03e06370ee75f2386ffb46f54fa
Summary:
Previously, depending on the code path, the limit specified would not actually take effect. For instance, if we came in from debugmanifestcache, and attempted to populated the cache, we would use `systemawarecachelimit` when filling the cache, and the fixedsize limit specified by the user when pruning.
With this change, we unify the all the cache limit decisions to `fastmanifestcache`. If the user actually overrides the limit, we set the limit in `fastmanifestcache` and let that make the decisions.
We also change the definitions of limit in `hg debugcachemanifest` to:
1) >0 => it's the limit.
2) =0 => use systemawarecachelimit
3) <0 => no limit!
Test Plan: pass existing unit tests. there's a small change in the test output, because we always evaluate the limit now, plus we remove the test for limit=0, since it means something different now.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: trunkagent, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3544997
Signature: t1:3544997:1468281604:8f78f00ebf2afd8f3f1fbefbd82316b97cc4b193
Summary:
We need a fastmanifest object in order to size it. Once we know its size, we can make room in the cache.
This slightly affects one of the tests, as we request the manifest text earlier than we previously did.
Depends on D3537904
Test Plan: used in later diff.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters, quark
Differential Revision: https://phabricator.intern.facebook.com/D3538991
Signature: t1:3538991:1468280648:41c65d91529babe0559eac7b75509481adf2765f
Summary: We don't need this since we loosely lock the cache worker.
Test Plan: pass existing tests.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters, quark
Differential Revision: https://phabricator.intern.facebook.com/D3537355
Signature: t1:3537355:1468268248:ebf7de5bf5030b90039823e1111a073d7eacc447
Summary:
The shared cache needs to be completely g+ws so that all members of the group
can write to each directory in it. The old code only applied g+ws to the leaf
directories, so other users aren't able to write to non leaf directories (like
hgcache/7a/83beca8.../ others couldn't write to 7a/)
Test Plan:
Updated a test to view group permissions for the intermediate
directories
Reviewers: #mercurial, ttung, simpkins
Reviewed By: simpkins
Subscribers: lcharignon, net-systems-diffs@, simpkins, mbolin
Differential Revision: https://phabricator.intern.facebook.com/D3523918
Signature: t1:3523918:1467930221:452b11b56a2e69896bf8d2cd0acd7131b41f90d8
Summary:
Previously, the history repack logic would stop traversing history for a given
filename once it encountered a rename. This isn't quite right, since the history
could eventually be traversed back to the original file, where we'd need to
continue processing. So now we check for when the copyfrom becomes the filename.
Also, if the copy source file and the copy target file have two nodes with the
same value, we would not process the one in the copy target (since it was marked
do not process). We fix this by explicitly checking if the node is one of the
known entries in the file being processed.
Test Plan: Added a test
Reviewers: #mercurial, ttung, mitrandir, rmcelroy
Reviewed By: mitrandir, rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D3523215
Signature: t1:3523215:1467828169:bd487c8f296352c1a1b9355cb55f9001bd5e19a9
Summary:
Before this patch we were doing two mistakes in the ratio computation.
- We were not recording global cache hit or miss, it was always a hit because
a function is truthy is python
- We were not deduping cache miss for the same manifest multiple times
This patch fixes these two mistakes.
It also changes the logging of cache hit and miss to include the name of the
operation that triggered the cache hit or miss (diff or filesnotin).
Test Plan: Test output changed and commented
Reviewers: durham, ttung
Differential Revision: https://phabricator.intern.facebook.com/D3484658
Summary:
Before this patch we were using config instead of configbool for reading
the debugmetrics config causing "False" to be evaluated as a truthy value
for the config. This patch fixes the issue and sets the config to false for
some of the tests to reduce the noise of the output.
Test Plan: Tests pass
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3524501
Summary:
There are two parts to add logging to blackbox. Logging events
and changing the configuration of blackbox. This diff does the former.
We don't reuse all the metrics event as:
- blackbox does not support wildcard for events like fastmanifest-*
- they are not meant to be human readable, they are for performance logging
We instead log a few new events:
- What triggered a caching operation?
- What command we run to cache the manifest?
- What revisions are to be cached?
- What revisions are actuallt cached?
- Do we overflow the cache?
This will allow us to understand all the cases where caching didn't trigger.
Like in t11877434.
Test Plan:
Test output does not change
Verified manually by changing the blackbox configuration that events get logged:
CHGDISABLE=1 hg book food --config blackbox.track='command, commandfinish, commandexception, exthook, fsmonitor, pythonhook, fastmanifest' --config extensions.fastmanifest=/home/lcharignon/facebook-hg-rpms/fb-hgext/fastmanifest --config extensions.blackbox=/home/lcharignon/facebook-hg-rpms/hg-crew/hgext/blackbox.py
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3504158
Tasks: 11877434
Summary: D3505464 modified the kwargs, but the test output was not updated.
Test Plan: passed tests.
Reviewers: lcharignon, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3506179
Signature: t1:3506179:1467353060:7c7cf9fd51257f5f5aa037a779618cf6512c1dd0
Summary:
reflog is dead, long live journal! journal is at feature parity and reflog can
go. I've marked this as a move to help find what happened in history.
Test Plan:
Run all the tests to see if anything broke:
$HGSRC/tests/run-tests.py -l
Reviewers: #sourcecontrol, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3498613
Tasks: 10804988
Signature: t1:3498613:1467232574:75c80b0223c101236b4d4e0f27f46f72797ebe90
Summary:
chg runs extsetup and uisetup with dummy ui objects. Before this
patch we were relying on ui objects in extsetup and uisetup. After this
patch, we move the logic to reposetup. We do things a little differently
because reposetup can be called multiple times and extsetup and uisetup are
called only once.
This caused one problem for `hg clone` in the test as reposetup is not
called for clone operation. I adapted the test a little bit to cover this case.
Test Plan: Modified tests pass
Reviewers: mitrandir, quark, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3499176
Signature: t1:3499176:1467244335:a70d718725c1d19ab6b6feb4558a8ffd1a020ab2
Summary:
Rather than put everything into one journal file, split entries up in *shared*
and *local* entries. Working copy changes are local to a specific working copy,
so should remain local only. Other entries are shared with the source if so
configured when the share was created.
When unsharing, any shared journale entries are copied across.
Test Plan:
Run all journal tests:
$HGSRC/tests/run-tests.py -l test-journal*
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3481745
Tasks: 10804988
Signature: t1:3481745:1467215718:3b6666c2b908cea279f181f5a9310c0093d0c0fe
Summary:
Note that now the default action for `hg journal` is to list the working copy
history, not all bookmarks. In its place is the `--all` switch which lists all
name changes recorded, including the name for which the change was recorded on
each line.
Depends on D3458115
Test Plan:
Run all journal tests:
$HGSRC/tests/run-tests.py -l test-journal*
Reviewers: #sourcecontrol, rmcelroy, durham
Reviewed By: durham
Subscribers: durham, mitrandir, rmcelroy, ttung, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3462793
Tasks: 10804988
Signature: t1:3462793:1466703746:8ee2b9822e39a458d64d30b160de3ecb77c623e7
Summary:
This change would have to go into remotenames directly when hg journal itself
is added to Mercurial core.
Depends on D3458077
Test Plan:
Run the included test-journal*.t tests
$HGSRC/tests/run-tests.py -l test-journal*
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3458115
Tasks: 10804988
Signature: t1:3458115:1467217834:1f8421da38280ab333faa7142118de1ef0e1aed4
Summary:
This is slated to be the upstream version of reflog. It is being hashed out
here to accellerate moving from reflog to journal.
This initial revision only supports bookmarks; later diffs add remote bookmarks
and working copy changes. As such the storage format is built to support those
usecases but the commandline usage is still geared towards only needing to list
bookmarks for now.
We'll replace reflog.py with journal.py altogether, but due to the fine details
of our RPM / chef config duality reflog.py needs to stick around until we
rolled out the RPM everywhere and updated chef to follow.
Test Plan:
Run the included test-journal.t test
$HGSRC/tests/run-tests.py -l test-journal.t
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3458077
Tasks: 10804988
Signature: t1:3458077:1467215042:d3fbba283901de6abee5d2b28a43ed9293627795
Summary: ui.log expects msg[0] to be the format string, and msg[1:] to be the arguments.
Test Plan: extend test-sampling to cover the case of a formatting string + args. pass test-sampling.t
Reviewers: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3479711
Summary:
Since metrics doesn't send a string, let's just stub out the call.
The alternative is to put a '' in every `recordsample()` call.
Test Plan: pass existing tests.
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3479138
Tasks: 11900487
Signature: t1:3479138:1466787619:910917093cf0361ed181af5597ab702b936f8d1e
Summary:
1. verify that we cache entries on background.
2. verify that we don't fill the cache when we can't take the lock.
Test Plan: passed existing unit tests.
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3471355
Signature: t1:3471355:1466719450:191070cd2d5ef93e0e1a6925a04d1a885431cbb4
Summary: --created is the opposite of --deleted.
Test Plan: meh. used in later diff.
Reviewers: lcharignon, mjpieters
Reviewed By: mjpieters
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3478780
Signature: t1:3478780:1466759469:47a3af68df2c8d550aa654f135d4f0fcf6df7851
Summary: It's actually most relevant to least relevant. Add a docblock.
Test Plan: pass unit tests
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3478616
Signature: t1:3478616:1466787566:b38eceddf91c3e88341f9173e07244ac33ab6345
Summary: This allows us to build tests to verify the behavior of background cache fills.
Test Plan: used in next diff.
Reviewers: mjpieters, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3471343
Tasks: 11683504
Signature: t1:3471343:1466698914:93d15b2d4879b9d8a91aeab62fcf119f24de3c1c
Summary:
Everything will be synchronous within the process. We will do background work in a separate process.
Depends on D3468827
Test Plan: updated tests to remove the one background task.
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3468828
Tasks: 11683504
Signature: t1:3468828:1466694717:556968a10717c3dd7ce403593cc6a1e97067633f
Summary:
This extensions prevents mercurial from being ctrl-c'ed by warning the
user on the first SIGINT. The subsequent interrupts are handled normally.
Test Plan:
see tests
(the tests are lacking the scenario of double ctrl-c - any suggestions on how
to do it in nice way inside the test framework welcome)
Reviewers: #sourcecontrol, durham, wez, quark, akushner, rmcelroy
Subscribers: lcharignon, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D2936488
Summary:
Directly execute the xargs command, rather than using a shell. This is
slightly more efficient, but also ensures we won't have security problems or
other issues based on the user's grep.command config setting. Even if it
contains shell redirects, semicolons, or other special characters, we will no
longer pass them to a shell to interpret them.
We do use shlex.split() go process the grep.command setting, to allow the user
to specify a command plus arguments.
Test Plan: Included new unit tests.
Reviewers: #sourcecontrol, durham, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: akushner, rmcelroy, sid0, net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3376617
Signature: t1:3376617:1464841542:f58fc0a05d62b649edc70bc261009875385186c9
Summary:
Previously grep only searched in the current directory, even if you gave it a
pattern like "../". This fixes the code to correctly honor the arguments, and
to accept mercurial file patterns just like most other mercurial commands.
This does change the file parsing behavior somewhat: previously "hg grep" would
treat all arguments as glob patterns. Now you explicitly have to say
"glob:<pattern>" to get glob matching. However, I suspect this won't impact
most users, since they would normally let their shell perform glob expansion
before the arguments are given to mercurial.
Test Plan:
Included new unit tests for grep, and updated the existing tests that were
affected by the change in glob behavior.
Reviewers: #sourcecontrol, durham, carenthomas, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: quark, mwilliams, matthieu, meyering, yfeldblum, rmcelroy, net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3376605
Signature: t1:3376605:1464841648:98bfb47866dec802ca34f55c821796f36532211f
Summary:
This patch fixes various things around the sampling extension to
match what our wrapper expects to see. See detail in the test.
Test Plan:
lcharignon@XXX fbsource cat ~/.hgrc
...
[extensions]
sampling=
[sampling]
key.fastmanifest-cachehitratio=table_blah
key.fastmanifest-trigger=table_blah
filepath=/dev/shm/samppath
lcharignon@XXX fbsource hhg book foo
lcharignon@XXX fbsource cat /dev/shm/samppath
{"category": "table_blah", "data": {"source": "bookmark", "metrics_type":
"fastmanifest-trigger"}}\0{"category": "table_blah", "data": {"ratio": -1, "metrics_type":
"fastmanifest-cachehitratio"}}\0
Reviewers: ttung, durham
Differential Revision: https://phabricator.intern.facebook.com/D3466719
Summary:
This patch adds a test that uncovers logic bugs in the cache. It also
adds fixes for these bugs.
Test Plan:
Add a new test and ran existing tests, observed no change for
exisiting test.
Reviewers: durham, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3450430
Summary: This patch fixes the torulemethod of the exec action to match upstream.
Test Plan: Add a test that failed before the codechange
Reviewers: mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3459588
Summary:
The upstream change has now been *properly* landed at
https://selenic.com/repo/hg//rev/6ee50bdeeb32.
Once again, Mea Culpa, yesterday I apparently was in clowntown mode with these
patches.
Test Plan:
Run all affected tests:
$HGSRC/tests/run-tests.py -l test-pull-createmarkers* test-pushrebase* test-pushvars*
Reviewers: #sourcecontrol, ttung, quark
Reviewed By: quark
Subscribers: mjpieters, durham
Differential Revision: https://phabricator.intern.facebook.com/D3462832
Signature: t1:3462832:1466513182:58d6187c04e0e567a026aba90bf753c76f4949a7
Blame Revision: D3460645
Summary:
e61bd9d5e8ca requires a change to core that has been accepted but not yet
pushed. This breaks the build, so let's revert it until upstream pushes the
dependency.
Test Plan: Ran the tests
Reviewers: ttung, mjpieters, #sourcecontrol, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3460645
Signature: t1:3460645:1466465904:cd521a989e475f9987237307a7d4d672d58ba615
Summary:
Before this patch, we were not importing mercurial.error, this was
causing a crash when calling error.Abort. This patch adds the missing import.
Test Plan: Tests pass, and add a new test
Reviewers: durham
Differential Revision: https://phabricator.intern.facebook.com/D3457086
Summary:
Now that my upstream patch for filedescriptor class access has landed the helper function to bypass the descriptor protocol can go altogether.
Depends on D3454878
Test Plan:
Run affected extension tests:
$HGSRC/tests/run-tests.py -l test-sqldirstate-* test-reflog* test-sparse*
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3456368
Summary:
There were 3 clowntowny versions of the wrapfiledescriptor function, 2 being
dangerously wrong about how they scan the class hierarchy. All because
`classobj.descriptorname` invokes the descriptor protocol and the filecache
descriptor is not set up for that access.
Instead of one function to do it all, use one utility function to retrieve
the descriptor object from the class __dict__ to avoid the descriptor protocol
kicking in, and use mercurial.extensions.wrapfunction to do the actual
wrapping.
I've sent a patch upstream to fix accessing the filecache descriptor on the
class; if accepted we can drop the getfilecache utility function altogether,
leaving only the wrapfunction calls.
Test Plan:
Run affected extension tests:
$HGSRC/tests/run-tests.py -l test-sqldirstate-* test-reflog* test-sparse*
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3454878
In this race condition test, occasionally the second invocation would actually
obtain the lock before the first. This meant that the first repack would fail
with an error message while the second would exit with 0, resulting in the test
output changing slightly. Let's introduce a slight delay before the second
invocation to prevent this from happening.
Summary: This will be useful to refactor the cache
Test Plan:
We change the python test to give a vfs that makes sense, it will
be useful in the next patch when we refactor the cache. Indeed, with the new
cache we use systemawarecachelimit by default instead of no limit
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3427796
Summary:
Before this patch, if we decided to remove entries A,B,C where A is
more recent than B, which in turn is more recent than C. We were removing
in order A,B and C. This patch changes the order to C,B and A and will make
the next patches simpler.
Test Plan: Tests are updated accordingly
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3427795
Summary:
When refreshing cache entries, it is useful to know what delay we
introduce. This patch adds a debug line showing that information.
Test Plan: tests are updated accordingly
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3427794
Summary:
Before this patch, it wasn't clear from hg debugcachemanifest --list
which manifest were older and newer or what rev were actually cached. This
patch improves the output of hg debugcachemanifest --list --debug to show
this information.
Test Plan: add debug output to the test
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3427791
Summary:
Before this patch we were returning the cache entries alphabetically.
This patch changes the behavior to return the entries sorted by date and
alphabetically. This will be used to simplify the code for pruning entries
in furter patches.
Test Plan: test output changes
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3427789
Summary:
Before this patch we were stopping execution when the lock couldn't
be taken. This was problematic when running hg command for someone else's repo
(for example to debug problems). This patch allows us to continue the execution
of the command when that happen, except we won't use fastmanifest.
Test Plan: This is reflected in the test output
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3422458
Summary: This could break people using fastmanifest in someone else's repo
Test Plan: tests pass, not sure how we normally test permission stuff
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3420870
Summary: It can't find those extensions now that we removed the extensions from the system pythonpath. Therefore, we need to load it from the source directory.
Test Plan: pass PYTHONPATH=~/work/mercurial/facebook-hg-rpms/remotenames/:~/work/mercurial/facebook-hg-rpms/mutable-history/hgext python ~/work/mercurial/facebook-hg-rpms/hg-crew/tests/run-tests.py -j32 test-fbhistedit-stop.t
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3419503
Signature: t1:3419503:1465590764:23e348efee30cabbbff232b7457d5820125bd6ad
Summary: printf appears to be more platform agnostic than echo -n, which is not supported on bsd/mac.
Test Plan: run PYTHONPATH=~/work/mercurial/facebook-hg-rpms/remotenames/:~/work/mercurial/facebook-hg-rpms/mutable-history/hgext python ~/work/mercurial/facebook-hg-rpms/hg-crew/tests/run-tests.py -j32 test-catnotate.t
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3419500
Signature: t1:3419500:1465590785:a34da9af73c64d9edef66a7b4a6ed3abd885e5c7
The previous commit fixed a bug where copyfrom data was represented incorrectly
in the local .hg/store/data remotefilelog blobs when the ancestor data was read
from a pack file. This commit adds a test for that situation.
Summary:
Now that repack can clean up old remotefilelog blobs, let's have it also delete
any empty directories that get left behind.
Test Plan: Updated an existing test to cover it
Reviewers: mitrandir, lcharignon, #sourcecontrol, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar
Differential Revision: https://phabricator.intern.facebook.com/D3385546
Signature: t1:3385546:1464972782:5ca63cf0a5589bb8a537957f50b4bc5ec4e0f0f5
Summary:
Previously a bunch of different places accessed the cachepath through ui.config
directly. This is a problem because we need to resolve any environment variables
in the path, and some spots didn't do this. So let's unify all accesses through
a helper function that takes care of the environment variables.
Test Plan: Added a test
Reviewers: mitrandir, lcharignon, #sourcecontrol, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar
Differential Revision: https://phabricator.intern.facebook.com/D3385583
Signature: t1:3385583:1464971813:5b9ee5ed3d6ff9f1a78cb9e0269e433844758c9d
Summary:
As discussed, we want to make sure we have a good SocketServer for Python 2.6,
which is unsupported by PSF but still being widely used by our users.
This diff adds a new extension that tries to fix buggy Python standard modules,
namely the SocketServer. It fixes a critical race condition for Python 2.6
(https://bugs.python.org/issue21491) and the zombie process issue for both
Python 2.6 and 2.7 (https://bugs.python.org/issue11109).
A simple test is added to check zombie processes are no more.
Test Plan:
Run the newly added test, `test-patchpython.py` with both Python 2.6 and 2.7.
Comment out the `import patchpython` line in the test and make sure it fails
on Python 2.6 and 2.7 with `AssertionError: zombie process detected`.
Reviewers: #mercurial, ttung, mjpieters
Reviewed By: mjpieters
Subscribers: simonfar, durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3384909
Signature: t1:3384909:1465319491:0ba64b496186712651b7ee6bd502d1e2ccac57d1
Summary: Add code to log metrics for fastmanifest, also handles aggregate
metrics. We reuse ui.log for logtoprocess. We also refactor diffs and
filesnotin to make it easier to log cache hit and miss ratio.
Test Plan:
Replaced ui.log by ui.status and aggregated the field, check
that I see metrics getting logged.
Reviewers: ttung, durham
Differential Revision: https://phabricator.intern.facebook.com/D3368504
Summary: This mostly replicates the logic in `manifest.py::manifest::add(..)`, except it interfaces with the fastmanifest cache instead.
Test Plan:
used `hg debugcachedmanifest` to populate the cache, then did a commit with `pdb.set_trace()`.
added a correctness test that exercises the code path.
Reviewers: simonfar, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3397910
Tasks: 11294278
Signature: t1:3397910:1465407138:60c7b696b4bafa153c8076ab58c53bf71a9fcfd4
Summary:
Before this patch, all the events logged by the sampling extensions
were dumped to the log file. With this patch, one can specify the relevant
events to log as well as a value to be added for each event. This will be
useful to specify what logging table to use for each of the logged events.
More information added to the documentation accordingly.
Test Plan:
The test changes accordingly and we add events that gets stopped
by the filter to make sure that the filter works.
Reviewers: ttung, durham
Differential Revision: https://phabricator.intern.facebook.com/D3387446
Summary:
We used to have test code to make the test deterministic. Now we
enforce the same constraint in code and that code is not necessary anymore.
We change the test output to match the new, deterministic output.
Test Plan: Ran the tests 80 times and checked that the output is stable
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3387259
Summary:
This will allow us to know what revisions are actually cached vs
revisions that would be cached by a trigger.
Test Plan: Add new tests
Reviewers: ttung, durham
Differential Revision: https://phabricator.intern.facebook.com/D3385955
Summary:
`lrucachedict`'s `get` method is a lower-level API that retrieves a _lrucachenode object, not what we want. the idiomatic construction is:
```
if key in lrucachedictobj:
do_something(lrucachedictobj[key])
```
Test Plan: hg amend with fastmanifest works. wrote a test that failed before this diff, passed after.
Reviewers: #mercurial, lcharignon, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3384502
Tasks: 11569523
Signature: t1:3384502:1465004304:8dfee6a4d0b2f6bb39262a310744458838cb0bf3
Summary:
This patch adds a new extension: sampling. It allows us to collect
aggregated data to a log file for each command that runs.
We will use it with our wrapper to collect data on sqldirstate and fastmanifest.
The workflow:
- Wrapper creates a log file, passes it to mercurial through an environment variable
- Mercurial writes data to this log file
- The wrapper parses the log file and upload its content to a data logging backend
Test Plan: Add a test
Reviewers: mitrandir, wez, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3360988
Summary:
BSD version of head does not support -n0 and it looks like the only purpose of
using it here is to hide stdout output (and leave stderr).
Test Plan:
Ran test-tweakdefaults.t on both Linux and OSX, made sure that this particular
part of the test does not fail anymore.
That said, test-tweakdefaults.t still fails on OSX, but that requires more
investigation and my 5-minute look did not reveal the reason.
Reviewers: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3378544
Summary:
Before this patch, we were passing None as ui for hybridmanifest's
constructor in test-fastmanifest.py. With this patch, we pass a dummy ui object.
Therefore, it removes the need for None guard in hybridmanifest and simplifies
the code.
Test Plan: Tests pass
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3374069
Summary: We got all we wanted from this.
Test Plan: meh.
Reviewers: #mercurial, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3373068
Signature: t1:3373068:1464810628:ebfa809a0a02d6814c153c1b47354aae9a98dde1
Previously, background repacks would only repack pack files, which meant there
was no automated way to repack loose remotefilelog files without manually
running 'hg repack'. This allows incremental repacks to also pack the loose
files.
It also changes the config knob for background repacks, so we can enable pack
file usage without the server having to support it just yet.
Summary:
Instead of refreshing the cache at the time of the trigger, accumulate the repos we need to refresh, and do so once at the end.
This also allows us to remove the hacky once-per-process gating.
Test Plan: pass most of the existing tests, update the remotenames test.
Reviewers: quark, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3363061
Tasks: 11475606
Signature: t1:3363061:1464465376:574c694d345b61109dca2c9e0b9fee0d8f742e46
Summary: dirstate changes are overly broad. we really only want the trigger if we make a new commit.
Test Plan: will update tests after this series of commits, as the tests are not really exercising the code paths we want.
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3363013
Tasks: 11475606
Signature: t1:3363013:1464465629:8aad5cbb5f357e751a11aab593347327c50314b5
Summary:
This is neccessary for scmprompt to work. I added a config knob because we
may want to disable it if we want to do paranoid mode at some point.
Test Plan: ran tests
Reviewers: #mercurial, durham
Differential Revision: https://phabricator.intern.facebook.com/D3356646
Summary: I based my code on the generaldelta upgrade which also occured on pull.
Test Plan: test will follow in next comit
Reviewers: #mercurial, durham, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3356590
Summary:
This is somewhat insulated against potential future design changes in chg, as we record the last pid to execute `cachemanifestfillandtrim`, and only suppress it if the pid is unchanged.
Notice the unit test no longer does extra calls to populate the cache.
Test Plan: passed unit tests
Reviewers: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3357697
Summary: When --debug is turned on for commands, there's a ton of spew. This makes it easier to spot the parts I care about.
Test Plan: update tests
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3357974
Signature: t1:3357974:1464365624:83edc815109db8533465991413a7c7741f1056a8
Summary: We want to wrap the fastmanifest cache priming code in a lock, but we want it to be easily stealable. If the lock is more than X seconds in age, we just assume it's stale and steal it. We remember that we stole it so we don't blow away the lock, but we do update the time of the lock so another process doesn't try to steal the lock.
Test Plan: write test to cover basic functionality (mutual exclusivity). write test to cover stealing, and that only one thief is permitted at a time (subject to race conditions).
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3343478
Tasks: 11385124
Signature: t1:3343478:1464304447:18658ebea60f98bfda0f034414991ffd6c334ca7
Summary:
This test is mostly a copy of test-shelve.t from mercurial with
minor changes (removing the hooks usage).
The normal tests for shelve unfortunately are using out of proces
hooks which doesn't work good with sqldirstate. As shelve is
important extension for us we want to test is with shelve anyway.
Test Plan:
test passes on clowncopter with this patch applied:
http://patchwork.serpentine.com/patch/15207/
Reviewers: #mercurial, wez, quark, durham
Reviewed By: durham
Subscribers: quark
Differential Revision: https://phabricator.intern.facebook.com/D3350194
Signature: t1:3350194:1464260281:c85a1d5ae50d9488a8bcdb552343f466bebb9e05
Summary: This combines D3351047, D3351086, and D3351137 into one diff. This allows mercurial to better remember where things came from since these are all hg cp commands followed by some edits. However, they were separated into multiple diffs for ease of reviews.
Test Plan: run unit tests
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3354405
Signature: t1:3354405:1464291057:da140a02b9c87186b1dd9f03f8236d5996319fff
Summary:
Before this patch, we were caching in the background in remotenames
test. This could result in a race condition making the test non deterministic.
Test Plan: Test changed
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3353406
Summary:
As discussed it is not relevant as the tests cache so little it should
not make the test non-deterministic.
Test Plan: The test change accordingly since the limit is no longer None
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3353394
