Summary: `PyTuple_Pack` does not steal the reference, so we need to decref.
Test Plan: valgrind is my friend.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, lcharignon, mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3698714
Signature: t1:3698714:1472175554:d5f1cde0ba1571357d746a45ab9f4dbd0d0ccb6e
Summary: No longer needed.
Test Plan: make local && clion build && make unit tests.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3780786
Signature: t1:3780786:1472255290:cdf238f772234f8a2e15d69dca6489b6b977511d
Summary:
buffer.h gained the ability to deal with non-char-sized buffers when I built cdatapack. We need to update the callers in ctreemanifest to be aware of this. Most of this is done with macro magic.
Some functionality was dropped from cdatapack's buffer.h (macro definitions to deal with paths). Those are moved to path_buffer.h
Test Plan:
make local && clion build.
pass cfastmanifest unit tests.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3780767
Signature: t1:3780767:1472255278:40a19edfd171df5804e9cdfa4444d5c6386f00e8
Summary: One giant CMake file. Only I use CLion, so i don't think this affects anyone else.
Test Plan: make in clion. run cfastmanifest unit tests.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3780573
Signature: t1:3780573:1472250750:ece55fb66a656624335c8432ab3c8334699ea36e
Summary: Ubuntu 14 apparently requires `_BSD_SOURCE` to be defined for nteoh64 to exist. Also, MAP_FILE is not part of POSIX mmap spec, so is not always available. Fortunately, on the platforms we support (linux and osx), it appears to be implied by default.
Test Plan: make local
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3780057
Signature: t1:3780057:1472246090:100c38f613e67dc1c56ab6ef55a3cc7c4f703497
Summary: When a ManifestEntry is destroyed, we reap the heap-allocated resources it holds. However, this (erroneously) happens when we create a ManifestEntry and add it to the list of children. What we really need is a move constructor from C++11. The workaround is to initialize a blank ManifestEntry, add it to the proper location, and then populate it afterwards.
Test Plan: addChild (in a later diff) no longer tramples over memory it no longer owns.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3772915
Signature: t1:3772915:1472174144:a4baac5cb5f6e01a38042c5c6cd92570c8f8e100
Summary: This gives us a bit of flexibility when we call this method, as references cannot be reassigned.
Test Plan: compiles
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3772844
Signature: t1:3772844:1472174021:565d7f25a27c2fd78ce14b86b5bc499c74d47976
Test Plan: haven't used this yet, but it makes a whole lot of sense.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3770366
Signature: t1:3770366:1472173969:b83e10e27a63b863a3566d2df7cc861417b8208e
Summary:
1. `children()`, which returns the number of children the manifest has.
2. `removeChild()`, which removes the entry that the iterator references.
Test Plan: used in later diffs
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3770363
Signature: t1:3770363:1472173823:737043b4613a669b402dcfd7921ac55278c3ece9
Summary: The strings are not necessarily null-terminated and length needs to be considered.
Test Plan: used in later diff to find a path.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3770358
Signature: t1:3770358:1472173786:9a2f681c90476aafd481c301ff65ac8b199214ec
Summary:
appendbinfromhex appends the binary representation of a 40-byte hex string onto a std::string. This allows us to reuse a std::string rather than to allocate a new one every time.
Also:
1. converted binfromhex to use this method.
2. updated the docblocks to actually reflect reality.
Test Plan: `make local && cd ~/work/fbsource && PYTHONPATH=~/work/mercurial/facebook-hg-rpms/remotefilelog:~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/remotenames/:~/work/mercurial/facebook-hg-rpms/lz4revlog/ /opt/local/bin/python2.7 ~/work/mercurial/facebook-hg-rpms/hg-crew/hg --config extensions.perftest=~/work/mercurial/facebook-hg-rpms/remotefilelog/tests/perftest.py testtree --kind flat,ctree,fast --test fulliter,diff,find --build "master~5::master"`
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3770048
Signature: t1:3770048:1472105520:eac79a42360ebfa258519346b68fc4541c2dbb7c
Summary: In most cases, this is pretty straightforward. The only unusual case is `_treemanifest_find`, which does the actual resolution of the root manifest (unlike diff and iter).
Test Plan: make local
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3763324
Signature: t1:3763324:1472173572:3dcda9b318ad818d2f51e8c3472c7770739faafe
Summary: This more accurately describes its purpose.
Test Plan: simple refactor, so thus just make local
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3763323
Signature: t1:3763323:1472104318:b48246777db0527c7022066438c211f54c88703e
Summary: It makes a destructor possible.
Test Plan: make local
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3763322
Signature: t1:3763322:1472104242:e76ef1943c2082ddecff872c6472de4706505922
Summary:
I've seen the failures like https://phabricator.intern.facebook.com/P56599278
which were the result of "enforce_root_files" being set in our global watchman
config.
It appears that the global configs are being picked up only on server start.
Killing server during the test won't help because watchman is automagically
restarted in our dev environment.
The way to do it properly is to mimic
https://github.com/facebook/watchman/blob/master/tests/integration/WatchmanInstance.py
until we do that I've just put a .watchmanconfig to fix the test.
Test Plan: tests is passing now
Reviewers: #mercurial, ttung, durham, wez, zamsden
Reviewed By: wez, zamsden
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3772968
Signature: t1:3772968:1472158987:0e08c5e9f862ba3d74d016d051b852512d06e399
Summary: I ran tests with --noskip and found one more failure
Test Plan: test passing now
Reviewers: #mercurial, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3772978
Signature: t1:3772978:1472155449:1c5d8e16bd51d2b06f15432dd47c6fba48da3abd
You can jump to a patch by pressing its numeric index, but this obviously only works with up to 10 patches. Moreover, with a large number of patches the feature is dangerous, since an accidental number press will lose your place. While we could do something fancy and prompt for multi-digit input in the many-patch case, it doesn't seem worth the complexity, and simply disabling 'goto' seems good enough.
Test Plan: make local, see same errors that I saw in remotefilelog repo
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3777382
Tasks: 12855049
Signature: t1:3777382:1472216857:efc09d134edb63f36c47187f7d443319d4609d62
Summary:
This makes it possible to run all remotefilelog tests without others
It also avoids some issues with name collisions in the upcoming merge.
Test Plan: next commit is a merge and no conflicts in tests/
Reviewers: #sourcecontrol, ttung, durham, mitrandir, simonfar
Reviewed By: mitrandir, simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3764379
Tasks: 12855049
Signature: t1:3764379:1472217061:67a0cc8f1fc29f991be08fe965679535ff6df27a
Summary:
Copy of hackbranch hackathon project with a few changes.
The goal is to allow saving commits under certain bookmarks in the separate store. These bookmarks are called 'scratch' bookmarks and are defined in config file (for example, 're:scratch/.+').
Scratch commits are sent in a separate bundle2 part during push. During pull new bundle repo is created and requests are served using this bundle repo.
There are a few changes from hackathon project implementation.
# It requires no changes in mercurial core. It was done by wrapping `findcommonincoming()` function and changing `lookup()` wireproto method (see below).
# Instead of introducing `listkeypattern()` wireproto method, `lookup()` method and `pull()` command were wrapped. I did it because I thought that it will make code easier. It probably haven't made it easier, it's almost the same. I can change it back to `listkeypattern` if it's necessary.
The bad thing about current implementation is that scratch bookmarks are not downloaded. But in the previous implementation also haven't downloaded bookmark if remotenames is enabled.
There are a couple of problems:
1) As I mentioned before, scratch bookmarks are not downloaded
2) It's impossible to pull both scratch and non-scratch bookmark
Test Plan: python ../../../hg-crew/tests/run-tests.py test-bundlestore.t
Reviewers: durham, rmcelroy, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3750999
Tasks: 12479677
Signature: t1:3750999:1472052396:5d7828566863f0aa437cdc34439ec0759503c6c9
Summary:
This is to avoid merge conflicts in the upcoming merge
remotefilelog: rename tests/test-* to tests/test-remotefilelog-*
This makes it possible to run all remotefilelog tests without others
It also avoids some issues with name collisions in the upcoming merge.
Test Plan:
merge these in my sandbox
next commit is a merge and no conflicts in tests/
Reviewers: #sourcecontrol, durham, ttung
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3764371
Tasks: 12855049
Signature: t1:3764371:1472105685:49035af7e8efdd9daf43e5bc7eb02e3f169ce425
Summary:
Code was written by Laurent, Durham and Mateusz on hackathon.
Index stores metadata, store stores bundles.
This diff adds interfaces and simple filesystem implementations
Test Plan:
cd tests
py.test *.py
Reviewers: durham, rmcelroy, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3741963
Tasks: 12479677
Signature: t1:3741963:1472207457:c9e7fb6163ef3cd6fdc1c4bd5113c623169c3a7c
Summary:
764cd9916c94 recently introduced code that was unconditionally checking the
repo.includepattern and repo.excludepattern attributes on a local repository
without first checking if this is a shallow repository. These attributes only
exist on shallow repositories, causing "hg pull" to crash on non-shallow
repositories. This crash wouldn't happen in simple circumstances, since the
remotefilelog extension only gets fully set up once a shallow repository object
has been created, however when using chg you can end up with scenarios where a
non-shallow repository is used in the same hg process after a shallow one.
This refactors the code to now store the local repository object on the remote
peer rather than trying to store the individual shallow, includepattern, and
excludepattern attributes.
Overall this code does still feel a bit janky to me -- the rest of the peer API
is independent of the local repository, but the _callstream() wrapper cares
about the local repository being referenced. It seems like we should ideally
redesign the APIs so that _callstream() receives the local repository data as
an argument (or we should make the peer <--> local repository assocation more
formal and explicit if think it's better to force an association here).
Test Plan: Added a new test which triggered the crash, but passes with these changes.
Reviewers: ttung, mitrandir, durham
Reviewed By: durham
Subscribers: net-systems-diffs@, yogeshwer
Differential Revision: https://phabricator.intern.facebook.com/D3756493
Tasks: 12823586
Signature: t1:3756493:1471971600:9666e9c31bf59070c3ace0821d47d322671eb5b1
Summary:
To use linelog, we need to be able to translate between hg commit hashes and
linelog revision numbers. This diff implements such a revmap using the most
direct way.
The revmap also contains an extra "flag" for each revision, which will be used
to mark if the revision is in the main branch or not, to handle merge commits.
Test Plan:
`import revmap` from IPython and test its interface manually. Also have a
simple script to get some idea about its perf with 10000 revisions:
```
import contextlib, time, os, random, revmap, sys
@contextlib.contextmanager
def benchmark(msg):
sys.stderr.write('%s: ' % msg)
t1 = time.time()
yield
t2 = time.time()
sys.stderr.write('%f seconds\n' % (t2 - t1))
def randomid():
return ''.join([chr(random.randint(0,255)) for _ in xrange(0, 20)])
rm = revmap.revmap('revmap1')
with benchmark('insert 10000 random revisions'): # ~0.3 seconds
for i in xrange(0, 10000):
rm.append(randomid(), flag=1, flush=False)
with benchmark('writing to disk'): # 0.02 seconds
rm.flush()
os.rename('revmap1', 'revmap2')
with benchmark('loading'): # ~0.015 seconds
rm = revmap.revmap('revmap2')
```
Reviewers: ttung, #sourcecontrol, ikostia
Reviewed By: ikostia
Subscribers: ikostia, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3709706
Signature: t1:3709706:1471936489:0bbe35ed39a2af3f06e1000c4f9674149ad43995
Summary:
One place we are losing time in the profiling extension is when
our profiling thread is interrupted between two invocations of clock()
that are supposed to fetch the same time. Fix to use now = clock() and
use the same time in both places.
Test Plan:
Running with upstream and custom extensions, I no longer observe
suspended execution times large than accumulated execution times.
Reviewers: ttung, durham, mitrandir, #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3772529
Tasks: 12879683
Signature: t1:3772529:1472151936:192ada10ac4738755c8fd27278dc7ac90faa17be
Summary:
We need to fix tests due to de-facto HG_URL variable semantics change in
https://selenic.com/hg/rev/a5fd89db5549
Test Plan: run-tests.py test-pushrebase.t
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3771208
Signature: t1:3771208:1472141889:562c84ca92b91d2dbe60decea53349f53a9676c4
Summary:
The built linelog.so cannot be discovered automatically. Change `PYTHONPATH`
to make sure it can be found.
Also fix the issue that the rename change misses some places in the test.
Test Plan: `unset PYTHONPATH` and run the touched files.
Reviewers: #sourcecontrol, ttung, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3771583
Signature: t1:3771583:1472145156:d05a1bf0d5920dfd0131acd1998698f2dddae1f8
Blame Revision: D3763992
Summary: I'll later add most of your hackhaton hackbranch code to this project with a few small fixes
Test Plan: No need
Reviewers: durham, rmcelroy, mitrandir
Reviewed By: rmcelroy, mitrandir
Subscribers: quark, rmcelroy, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D3736089
Signature: t1:3736089:1471876504:98c4efc3bdf66d60f98eed11a3f13bae56fdc716
Summary:
This adds support for the optional ctreemanifest type. If the module is
available and if `fastmanifest.usetree` is enabled (default: False), we'll use
tree manifests before falling back to flat manifests.
Test Plan: Ran the ctreemanifest perf suite and the fastmanifest test suite
Reviewers: #fastmanifest, ttung
Reviewed By: ttung
Subscribers: ttung, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3754309
Signature: t1:3754309:1471929802:3ccba8ce6160b7a6d20b599626efd5876bd969f6
Summary:
This assert is meant to check that at least one source was provided to the
hybridmanifest. The old version was broken though.
The test requires an update because it attempts to construct a hybridmanifest
with no source.
Test Plan: Ran the tests
Reviewers: #fastmanifest, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3760229
Signature: t1:3760229:1472062113:695085444915cb15a93db7f317580dcffc2f115d
Summary:
This adds the `fastmaniest.usecache` config option (default: True) which lets
users disable the use of the fastmanifest cache without disabling the entire
extension. This will be useful in a future patch where we'll use this same
hybrid manifest infrastructure to allow swapping in tree manifests (even on
repos that don't want fast manifest caches).
Also contains a minior fix to an assert that didn't appear to make sense (I
think it should be asserting that at least one data source is provided).
Test Plan: Ran the ctreemanifest perf suite and the fastmanifest test suite
Reviewers: #fastmanifest, ttung
Reviewed By: ttung
Subscribers: ttung, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3754301
Signature: t1:3754301:1471990417:0cde06f271d10c7f0c839e4f644310a4f525f3f6
Summary:
This seems to be an elegant choice. It does not conflict with other names,
and it does explain the concept to some extent vividly.
Credit goes to [trousers at lobste.rs](https://lobste.rs/s/nws1uj/help_us_name_new_mercurial_feature/comments/u4tsw7#c_u4tsw7).
Test Plan: Run the existing tests.
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3763992
Signature: t1:3763992:1472049269:5bb9de3b7b3a0f0dd9587b952151fc38351bc822
Summary:
Some people think this command should be a part of "amend". If you check the
meaning of the "amend" English word, that makes sense. But if you look at
the actual "amend" command, there are significant differences:
1. smartfixup rewrites a stack of changesets, and can even delete
changesets, where amend only rewrites the working directory parent.
2. smartfixup is best-effort, i.e. does not guarantee that 100% of
user-requested modifications will be included, where amend will just take
100% (with "-i", it's 100% chunks selected by the user).
3. a lot of "amend" flags do not make much sense to smartfixup, like message
editing (designed to edit a single changeset), "--addremove", "--secret",
etc.
4. literally, smartfixup shares little code with the existing "amend" logic.
"amend" is part of "fbamend" or "evolve". this extension should not
depend on any of them.
So it's cleaner to be a separate command, not a part of `amend`.
However, I think it makes sense to have an option to satisfy those who want
to use "amend". So they can use "amend --related", "amend --fixups",
"amend --stack", "amend --auto" or whatever they choose. This diff adds such
a config option. We may also ship such a config option to make the command
easier for discovery.
Note the "amend" version is slightly different from the original smartfixup
command. The former targets basic users who expect amend to take all of
their changes, while the latter targets power users understanding what's
going on.
Therefore, the "amend" version will print extra information about what
changes are ignored, for example:
```
# changes not applied and left in working directory:
# M a : 1 modified chunks were ignored
# M c : unsupported file type (ex. binary or link)
# R b : removed files were ignored
```
To support the above change, `fixupstate.status` was added to avoid a second
"status" run and handles the "status" with interactive mode correctly. An
issue about symbolic links being added to `fixupstate.paths` was fixed by
the way.
Test Plan: Run the newly changed test.
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3760498
Signature: t1:3760498:1472052376:7ddbfe763c7327d044b0d471c31a58fcb1e21dac
Summary:
Sharing repo.changelog across threads is problematic in mercurial
since it may internally cache things, and clearing cache while another
thread is executing is not nice. In particular, code such as the following
is inherently raceful when sharing changelogs (or anything derived from
revlog) across threads:
if self._cache:
if self._cache[0] == node:
return self._cache[2]
cachedrev = self._cache[1]
Reworked things a bit to make sure the spawned threads have their own
newly created changelogs and member functions internally referencing them.
Test Plan:
No longer seeing random exceptions being thrown from local log
threads (which terminates iteration)
Reviewers: rmcelroy, #mercurial, ttung, quark, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3733592
Tasks: 12341014
Signature: t1:3733592:1471908572:877b56241e56b6ea92c76023460d4fbe80a4263d
