Summary: I wanted scopes, but python's contextmanager doesn't really provide a real scope. Derp.
Test Plan: run the test.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3862225
Signature: t1:3862225:1473895847:c2e437b6cb8ea2de9a94c6243de1cf731906d995
Summary:
When we remove a file, we need to remove all the empty directories that lead to that file. This test ensures that we are accomplishing this.
Note that when we put in the fix for '/' "feature", this test may no longer work.
Test Plan: pass unit tests.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3855366
Signature: t1:3855366:1473807483:66bd4650ee014e7da3fa62ef529af0467462aeb9
Summary: This way, we can have a true nullable (null, a.k.a., not present, and any character).
Test Plan: make local and pass tests
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3855340
Signature: t1:3855340:1473807404:54c1ae17c735665935a52b0140b58094f0b45a26
Summary: `initialize()` does it already.
Test Plan: run unit tests.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3869524
Signature: t1:3869524:1474225262:57ccfd92a4d02bc92674007bcafaeafedef413a0
Summary: D3842603 broke reads from datapacks because `.initialize()` creates a blank manifest and attaches it to the manifest entry. When we subsequently try to read the root, we don't properly actually resolve it to the manifest in the datapack. Instead, we use that blank manifest which is wrong.
Test Plan: `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/remotenames/:~/work/mercurial/facebook-hg-rpms/lz4revlog/ /opt/local/bin/python2.7 ~/work/mercurial/facebook-hg-rpms/hg-crew/hg --config extensions.perftest=~/work/mercurial/facebook-hg-rpms/fb-hgext/tests/perftest.py --config remotefilelog.fastdatapack=False testtree --kind flat,ctree,fast --test fulliter --build "master~50000::master" --revs 'master + master~5000'
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3862319
Signature: t1:3862319:1474225268:f01ef876d569bc09c1e0ace71492bbaae017404e
Summary:
Writing the revmap back to disk can be a bit expensive if the revmap is not
small. This diff optimizes `revmap.flush` so it does not write anything if
nothing changed.
Test Plan: Profiling data shows no more substantial `write` syscalls.
Reviewers: #sourcecontrol, zamsden
Reviewed By: zamsden
Subscribers: zamsden, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3837023
Signature: t1:3837023:1473954024:6af83074096517ab1d3d05cb90d091af6e0c5d02
Summary:
We are going to add `push` option to force commits to go to bundle store.
This is a preliminary refactoring. Add another experimental config param that sets whether this is a scratch or usual push
Test Plan: Run `test-infinitepush-bundlestore.t`
Reviewers: durham, rmcelroy, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3862604
Tasks: 12479677
Signature: t1:3862604:1473934819:3ead319a885d99dd9a215941b317540121739b32
Summary:
Upstream no longer has manifestctx derive from manifestdict; instead it uses a
read function. Let's update hybridmanifestctx to do the same.
Test Plan: Ran the fastmanifest tests
Reviewers: #sourcecontrol, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3863656
Signature: t1:3863656:1473878502:6fb01f9f23fe64aafdc366a7ae98d510f3fb7927
Summary: The newly added error.py will be used in other files.
Test Plan: A related test will be added later
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: ttung, stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3836304
Signature: t1:3836304:1473411906:ca110c8214d2b1a0f9ae29111d75baa54c072c19
Summary:
The revmap stores 2 kinds of revisions:
- Real revisions in the main branch that can be annotated directly
- Revisions in a merge changeset's p2 branch. They are not in the linelog
linear history but may appear in lines introduced by the merge changeset.
We call those revisions in a "side branch", outside the "main branch".
This diff implements `__contains__` to test if a revision is in the main
branch or not, it will be frequently used in upcoming code like calculating
the annotate information for a side branch, or finding a joint point when
doing a hybrid annotate.
Test Plan: A revmap Python test will be added later and cover this feature
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3827995
Signature: t1:3827995:1473324222:4f0d94a2d21c033ab78b3a5dd31254692d36a852
Summary:
There is no point in disallowing it.
The next step is to allow to fetch arbitrary many scratch bookmarks with one command.
Test Plan: Run `test-infinitepush-bundlestore.t`
Reviewers: durham, rmcelroy, mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3835502
Tasks: 12479677
Summary:
scratchbookmarks are not automatically pulled (because they are not showed in
listkeys). Let's record them after the actual pull. It introduces a race
condition. But this race condition is unlikely and shouldn't be harmful.
Test Plan: Run `test-infinitepush-bundlestore.t`
Reviewers: mjpieters, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3835414
Tasks: 12479677
Signature: t1:3835414:1473703998:554c890be50f732f34b2793160ee1abb26561e39
Summary: I could have sworn I ran this before I committed, but clearly not. Sadface.
Test Plan: actually ran them.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3853260
Signature: t1:3853260:1473723548:f360c6c4559641deb157e570df08a51a7edf8e21
Summary: Any modifying operation sholud set the `invalidate_checksum` flag, and as we unwind the stack, we clear the nodes if the flag is set.
Test Plan: not yet tested
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3842843
Tasks: 13268688
Signature: t1:3842843:1473709799:89a82ffade1e56fe64ccae50abecd3de4b46f6bd
Summary:
This adds a Manifest function that can produce a serialized version of the
manifest that is appropriate for writing into a store.
Test Plan:
Used it as part of a future diff that serialized manifests to a pack
file.
Reviewers: #fastmanifest, ttung
Reviewed By: ttung
Subscribers: ttung, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3837714
Signature: t1:3837714:1473454260:1c5864730744a560cbad4fc4ab5263ef4447f7dc
Summary:
This api will be used to tell the store that new data has been added and the
store should check disk for new data (like new pack files).
Test Plan:
Used as part of a future diff that generates new tree entries during
hg pull time.
Reviewers: #fastmanifest, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3837694
Signature: t1:3837694:1473454036:0f1f37064cadc0fe3dabc6ecc6e3f7b3319e3d56
Summary:
This implements the copy function, which will be used to create a copy of a
parent manifest when beginning to create a new manifest.
Test Plan:
Used in conjunction with future diffs to serialize a new treemanifest
into a pack file.
Reviewers: #fastmanifest, ttung
Reviewed By: ttung
Subscribers: ttung, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3837415
Signature: t1:3837415:1473453896:494fc856256764f14741c65258f354e08383f1f7
Summary:
While on Linux, `ssh` prompts `username's password:`, the OS X uses
`Password:`. Change `_shoulddisableecho` to include that case.
Test Plan:
Run `chg push ssh://localhost//tmp/foo` on an OS X system and check echo
is disabled for the password prompt.
Reviewers: #mercurial, ttung, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3841947
Signature: t1:3841947:1473440175:ea2c171a4bfda9d00b4de8c7c51bb00d0b7194f1
Summary:
`chg` has some special handling on `ui.system` - if it is simple, it will
be forwarded to the client (the `chg` process) and the new process will be
a child of `chg`, which will inherit the controlling TTY correctly.
Otherwise, it will be a child of a server-side worker process, which does
not have a controlling TTY. That's the case if `util.system` is used.
Test Plan:
Insert `x sleep 1000` in the histedit plan and confirm the `sleep`
process runs as a child of the `chg` process:
```
41876 13176 pts/14 Ss 03:17 0:00 \_ zsh
4196 732 pts/14 S+ 03:20 0:00 | \_ chg histedit
7572 664 pts/14 S+ 03:20 0:00 | \_ sleep 1000
```
Reviewers: #mercurial, ttung, rmcelroy, simpkins
Reviewed By: simpkins
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3846243
Tasks: 13295639
Signature: t1:3846243:1473480297:59aaf34cb3a52200321122dc7c054e2454cd3bd3
Summary:
The upstream changed the signature of `verify` in 9ec9fba6ea06
(`histedit: move constraint verification to the 'action.verify' method`):
- def verify(self, prev):
+ def verify(self, prev, expected, seen):
The upstream also moves the check about `action.node` around so
they become required and we have to set `self.node` in our histedit actions.
Test Plan:
Run histedit with the latest (8a658b) upstream code. Use `x sleep 1` and
ensure it works.
Reviewers: #mercurial, ttung, ikostia
Reviewed By: ikostia
Subscribers: ikostia, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3846190
Signature: t1:3846190:1473671179:97b238748e99a5868bb84614b89360e750b529a4
Summary:
Diff introduces `listkeyspatterns` wireproto method. In future it will be able to filter keys by pattern, but for now it has limited functionality:
1) it is able to find scratch bookmarks only by it's full name (no patterns)
2) and it doesn't filters non-scratch bookmarks.
Test Plan: Run `test-infinitepush-bundlestore.t`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3797800
Tasks: 12479677
Signature: t1:3797800:1473440073:61db6803235c53afbb3cdc72084aa35ce7d0ee63
Summary:
Removes are treated as set to None.
The API is not exactly correct, as manifestdict needs to implement size/getitem/setitem, but we can refactor in a subsequent diff.
Test Plan: pass unit test
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3836197
Signature: t1:3836197:1473363870:21a447e4bc17148193fde7ed4887927089e54930
Summary: Uses `walk()` to locate the element. Callback is super simple again.
Test Plan: used in later diff.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3836142
Signature: t1:3836142:1473363781:0384d2c0b7154d3acbbab436c12bbdb55a5a63a7
Summary:
Python binding is pretty straightforward. The test exercises a few simple paths: set, update, and conflict detection.
Not yet done: tests to set a deeply nested directory structure.
Test Plan: PASSED THE SIMPLE TESTS. YES!
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3832682
Signature: t1:3832682:1473363415:60240c6bbbcec82184b4588b83bbda750e8b77f1
Summary: This uses the `treemanifest.find(..)` interface to locate the desired tree entry and add/update a leaf node (a.k.a. file). The `set()` method is mostly setup and teardown logic for find, and the little unique code that needs to exist is in `set_callback()`
Test Plan: used in later diff.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: trunkagent, durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3832664
Signature: t1:3832664:1473459754:a258f42791b4c4d561abad043d8ad21ddb48fbed
Summary: @durham's changes to manifest.py landed, so we need to update where we plug into fastmanifest.
Test Plan: `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/lz4revlog/:~/work/mercurial/facebook-hg-rpms/remotenames/ /opt/local/bin/python2.7 ~/work/mercurial/facebook-hg-rpms/hg-crew/tests/run-tests.py -j32 test-fastmanifest*.{py,t}`
Reviewers: rmcelroy, durham
Reviewed By: durham
Subscribers: mitrandir, durham, mjpieters, lcharignon
Differential Revision: https://phabricator.intern.facebook.com/D3838608
Signature: t1:3838608:1473439470:1a689f60e72d351f1cf12d9e2cf1d2da9ab96bd9
Summary:
Infinitepush index implementation that uses mysql db to store data.
Three tables:
1) Node to bundle
2) Bookmark to node
3) All bundles (this is necessary to
find unused bundles.
Test Plan: Code was tested but tests are not included in the diff - see comments below
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3811247
Tasks: 12479677
Signature: t1:3811247:1473378520:bc23b3436aa2e618a7bfefd2d571257dfe8ace94
Summary:
logginghelper extension returns full path to the repo instead of just repo name.
It makes it hard to aggregate results per repo in scuba.
Use just repo name instead of repo root.
We can get repo name either from `paths.default` config value or
a basename of the repo root.
Test Plan:
Before:
{P56637657}
After:
{P56637661}
Diff
{P56637665}
Reviewers: rmcelroy, ttung, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3827912
Signature: t1:3827912:1473413932:5c38ca48e58a080c97b35610ee8889d66b4d5ab3
Summary:
We are going to log many different pieces of information in this extensions.
Therefore the `reporootlog` is not an appropriate name anymore.
Change it to `logginghelper`.
Test Plan:
Note that repo root is still logged instead of repo name. It will be changed in subsequent diffs
{P56637657}
Reviewers: rmcelroy, quark, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3840985
Signature: t1:3840985:1473410718:62d83f008f2973b7ed5266b0a5ebf67fe54b48c4
Summary: This is needed when we're updating a node that already exists.
Test Plan: used in later diff.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3832631
Signature: t1:3832631:1473362920:8a39f02ced91cc9328c57e416d7dfc9a704b0cc6
Summary: Internally, mercurial transacts in bin nodes.
Test Plan: used in later diff
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3832614
Signature: t1:3832614:1473359939:033cb1d944680287fb6563523c7782a80118b9e2
Summary:
flags should be == NULL if it's not present, not that it points to NULL.
Since the flags pointer isn't always set, we have to use the filename pointer to set the separator field ('\n').
Test Plan: used in later diff.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3832555
Signature: t1:3832555:1473359723:9f05dd353f0c1866232214cd0d489d1fb4573f51
Summary:
This implements the walk function, which is almost identical to iterkeys, except
it filters the result using a provided matcher.
Test Plan:
As before, not sure how to execute this from in mercurial, but it's
pretty small so we'll test it with the full mercurial test suite once we can do
edits.
Reviewers: #fastmanifest
Differential Revision: https://phabricator.intern.facebook.com/D3775631
Summary:
This implements the ability to iterate of the entries of the manifest. We reuse
the key iter, but add a boolean option for returning the full (path, node, flag)
tuple instead of just the path.
Test Plan:
I'm not sure of how to execute this via an hg command, so for now I'm
just going to
doitlive
Reviewers: #fastmanifest
Differential Revision: https://phabricator.intern.facebook.com/D3775619
Summary: This is needed when we execute a `set`, which comes in with a binary node, but is stored as a hex string.
Test Plan: make local; used in later diff.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3831923
Signature: t1:3831923:1473292766:bd4c1fdde356a8de5b4b74328e3805b7e1eae752
Summary:
1. remove unused `fetcher` variables.
2. std::string's second arg is a size_t, not a Py_ssize_t.
Test Plan: make local
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3830569
Signature: t1:3830569:1473292747:823b5c06490c45c0bfcc59373e7935af0c1b3630
Summary: Since a lot of accesses are coming from within the treemanifest class, I renamed the cached variable as rootManifest_DO_NOT_ACCESS_DIRECTLY to make people think twice before accessing it.
Test Plan: `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/remotenames/:~/work/mercurial/facebook-hg-rpms/lz4revlog/ /opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python ~/work/mercurial/facebook-hg-rpms/hg-crew/hg --config extensions.perftest=~/work/mercurial/facebook-hg-rpms/fb-hgext/tests/perftest.py testtree --kind flat,ctree,fast --test fulliter,diff,find --build "master~5::master"`
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3823297
Signature: t1:3823297:1473213565:54657eeecff35549c71dc0af91c71cca9d426ac1
Summary:
Use external bundles to store bundles.
Put and get binaries should be provided to upload/download bundles to/from external store (see `put_binary` and `get_binary` config options). Put binary should output a handle, that can be used with `get_binary` to retrieve bundle. `put_args` and `get_args` config options can be used to pass additional params to the binaries.
Test Plan:
Change `test-infinitepush-bundlestore.t` test file to point to external store and run it
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3803936
Tasks: 12479677
Signature: t1:3803936:1473123987:c34987ee459b2a66ee831574a32cdc8231043cb2
Summary: Add exceptions that will be thrown from bundle stores
Test Plan: Run test-infinitepush-bundlestore.t
Reviewers: durham, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3803928
Tasks: 12479677
Signature: t1:3803928:1473073448:32f6191ea0548c3d83cbaad100b6c8c48cbd8131
Summary:
The root node is now optional. If it's not specified, an empty tree is constructed.
Added an unit test to cover this code path.
Test Plan: `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/remotenames/:~/work/mercurial/facebook-hg-rpms/lz4revlog/ /opt/local/bin/python2.7 ~/work/mercurial/facebook-hg-rpms/hg-crew/hg --config extensions.perftest=~/work/mercurial/facebook-hg-rpms/fb-hgext/tests/perftest.py --config remotefilelog.fastdatapack=True testtree --kind flat,ctree,fast --test fulliter,diff,find --build "master~5::master"` still runs.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3803569
Signature: t1:3803569:1472759620:02c1f723fe3b7f4cdca9da6399fc2296c6d5a022
