Currently, normal LFS uploads are completely disabled during a p4fastimport,
and users must run an external SQLite-reading uploader app after the import.
Instead, allow normal LFS functionality to work, but users may override it to
do the SQLite thing.
Differential Revision: https://phab.mercurial-scm.org/D1307
When a file changed to include a flag and then the flag is removed we did not
correctly remove the flag. We now correctly set flags for all revisions.
Differential Revision: https://phab.mercurial-scm.org/D920
p4fastimport's reposetup will break other lfs commands or verify. Let's
limit that side effect to p4fastimport related commands. So they won't
affect users with p4fastimport enabled.
Differential Revision: https://phab.mercurial-scm.org/D801
We must correctly return the value from the function we are decorating.
Test Plan:
run tests
Differential Revision: https://phab.mercurial-scm.org/D767
Summary:
We are adding an option --base to define a starting point for a
new branch. We define --bookmark as the head of the branch (we
might introduce a --dest option in addition). This allows us to create
new branches for release branches and others.
Note that for some reason, `arc unit` is generating different test
results than running the run-test.py manually, causing branch updates in
different situations. Therefore we have to glob a bunch of tests.
Test Plan:
python ../../hg-crew/tests/run-tests.py test-p4fastimport-* test-check*
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, medson, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5769328
Signature: 5769328:1504763221:78eb81f32ec46fc56619d77537c2b1944e938466
Summary: Since a blob commit does not associate with a p4 changelist, while a regular commit mirrored a p4 change has a p4 changelist number, p4fastimporter needs to know how to handle a blob commit. The solution here is to get p4basecl number (which tells it which changelist number it's based on when creating this blob commit) from the blob commit node, and import from there.
Test Plan:
$ cd ~/facebook-hg-rpms/fb-hgext/test
$ python ../../hg-crew/tests/run-tests.py test-p4fastimport-blobcommit.t
$ python ../../hg-crew/tests/run-tests.py test-p4*
All tests should pass
Reviewers: #idi, davidsp
Reviewed By: davidsp
Subscribers: medson, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5544046
Tasks: 20141624
Signature: t1:5544046:1502405269:9f449550d60b9d29d8be464ab1cc98500bc60cad
Summary:
Perforce encodes certain characters in filenames (see
https://www.perforce.com/perforce/doc.current/manuals/cmdref/filespecs.html)
We must decode this before we add it to Mercurial.
Test Plan:
python ../../hg-crew/test/run-tests.py test-check* test-p4fastimport*
Reviewers: #idi, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, steaphan, medson, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5570533
Tasks: 20133877
Signature: t1:5570533:1502010300:0ef3ce0c228c61e23396b5a8ac887ac8c65812ff
Summary: This is a followup diff of this diff: [[ https://phabricator.intern.facebook.com/D5509536 | D5509536 ]]. It makes the ignored user and the constraint on time difference between a user commit and a p4fastimporter import configurable.
Test Plan:
$ cd ~/facebook-hg-rpms/fb-hgext/tests/
$ python ../../hg-crew/tests/run-tests.py test-p4fastimport-gitfusion-race-condition.t
$ python ../../hg-crew/tests/run-tests.py test-p4*
All tests should pass
Reviewers: #idi, davidsp
Reviewed By: davidsp
Subscribers: wlis, medson, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5531404
Tasks: 20141882
Signature: t1:5531404:1501866435:25686443d99a2b96f8e8a7fa9e3f660ba8c8393e
Summary:
Symlinks must not have newlines at their end. However reading the
content from Perforce, it might contain a newline, so we now strip it.
Test Plan:
python ../../hg-crew/tests/run-tests.py --keep-tmpdir test-p4fastimport-import-modes.t
Reviewers: #idi, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, medson, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5535514
Signature: t1:5535514:1501601075:20ce811a92d1d5b5a973e05db1d2110599c5ef52
Summary:
Writing multiple entries to a revlog at the same time can trigger a bug
in revlog caching when a revlog separation of inlined data happend. New
transaction postprocessing callbacks will cause a changelog read which
triggers this bug, therefore we force revlogs to be separated from the
beginning, as a workaround for this bug until it's fixed upstream.
Test Plan:
Run an import of 600 revisions on a test server. Saw failure.
Run import with patch appleid of 600 revisions on test server. Failured
disappeared.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mitrandir, rmcelroy, medson, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5535509
Signature: t1:5535509:1501613606:97a0a2dca19ae6e915d4777ec6b14d1e9db46242
Summary:
We overwrite existing transactions instead of appending them. So let's just
append them instead. This problem becomes apparent when we started separating
changelog data files from changelog index files which breaks the tests.
Test Plan: python ../../hg-crew/tests/run-tests.py test-p4fastimport-transaction.t
Reviewers: #idi, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mitrandir, rmcelroy, medson, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5535507
Signature: t1:5535507:1501613588:efeead423ca6d6fb50a4d4f83af68e8f87688912
Summary:
When a user commits changes to a git-fusion repo, it pushes commit to perforce as user git-fusion-user, and then update the commit's author to be the actual author. We want p4fastimporter to skip the git-fusion-user commit and import the commit by the actual author. This diff sets the basic workflow, where we specify the constraint to be commits by git-fusion-user that is less than 30 seconds old.
// TODO: make the constraint configurable.
Test Plan:
$ cd ~/facebook-hg-rpms/fb-hgext/tests/
// test race condition
$ python ../../hg-crew/tests/run-tests.py test-p4fastimport-gitfusion-race-condition.t
// test all
$ python ../../hg-crew/tests/run-tests.py test-p4*
Reviewers: #idi, davidsp
Reviewed By: davidsp
Subscribers: medson, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5509536
Tasks: 20141882
Signature: t1:5509536:1501521842:17dd767615b8218980eda68d4113433d30805d92
Summary:
When we are falling back to cvs tools to read content of a file, we still do
keywrod expansion through cvs `co` command. This patch disables the keyword
expansion.
Test Plan: run tests on test-p4*
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: medson, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5485123
Signature: t1:5485123:1500948788:fe4bc2fb8c50858ba7d8689bbf41d1de41b81000
Summary: Check if the client exists, and if not, exit the import. Added a test for non-regression.
Test Plan: running test-p4fastimport-import-badclient.t
Reviewers: davidsp
Reviewed By: davidsp
Subscribers: medson, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5405340
Signature: t1:5405340:1499965824:394077f0d148be2e5fef15902c69703eaf360bbb
Summary:
Perforce has a feature called [[ http://answers.perforce.com/articles/KB/3482 | keyword expansion ]].
When a file is marked +k or +ko and you have a string like $Id$, it get's expanded to $Id: SOMETHING$ on Perforce. When importing a file like this from Perforce into Mercurial we need to do the reverse, renaming $Id: SOMETHING$ to $Id$ (and equivalent for all other expansion types).
Test Plan:
$ cd ~/facebook-hg-rpms/fb-hgext/tests/
$ python ../../hg-crew/tests/run-tests.py test-p4fastimport-import.t
$ python ../../hg-crew/tests/run-tests.py test-p4*
Reviewers: #idi, davidsp
Reviewed By: davidsp
Subscribers: davidsp
Differential Revision: https://phabricator.intern.facebook.com/D5266323
Tasks: 19211063
Signature: t1:5266323:1497890444:a1cced5faa5e48c708c2721510952ca4d27d1bd4
Summary:
revlog uses transaction.find and transaction.replace in cases where a
datafile is separated from an index (>= revlog._maxinline). In these cases the
importer broke as we did not implement find and replace in filetransaction.
We implement the two methods. However we ned a save way to store the data
portion which find must correclty return. We cannot store json as some
serializations aren't bidirecitonal. We can't store pickle as \n is a common
separator in pickle, and we use that to separate our entries. So we use pickle
and then base64 encode it. This is cumbersome but works.
The fix is aimed to make things "work" and I want to explore options to better
implement find and replace.
Test Plan:
- Working on a test that correclty triggers inline -> separation.
- Deployed it to our test system with test data and see commits comming in.
Reviewers: #idi, #ovrsource_warroom, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D5189308
Summary: Upstream has deprecated cmdutil.commands() in favor of registrar.commands()
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5106486
Signature: t1:5106486:1495485074:0e20f00622cc651e8c9dda837f84dd84cc51099e
Summary:
We are adding a simple transaction that works only with filelogs but
allows for concurrent access from multiple workers. This allows for a proper
rollback in case of a failure in a worker process, which previously would result
in bad data in the repositoriy.
Test Plan: rt test-p4* test-check*
Reviewers: #mercurial, durham, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5070340
Signature: t1:5070340:1494958313:b10b1eac5b42b36d1a587c4ae1c95fc2f8b5ad35
Summary:
move p4fastimport under hgext3rd as that's where it belongs.
Also has the benefit that we package it up.
Test Plan: rt test-p4*
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5056664
Signature: t1:5056664:1494631758:4892f47922d8fbbd2c1f7793f3f29ee73fa8fa42
