Summary: second time's the charm
Test Plan: make local, wait for contbuild
Reviewers: #mercurial, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4955570
Signature: t1:4955570:1493230082:3493ac29ff8d04a98e50f83febc9dd827e9b4a7c
Summary:
this is failing the contbuild:
```
In file included from cstore/pythonutil.h:23,
from cstore/py-datapackstore.h:21,
from cstore/py-cstore.cpp:17:
cstore/../cstore/store.h: In member function const char* ConstantStringRef::content() :
cstore/../cstore/store.h:37: error: nullptr was not declared in this scope
error: command 'gcc' failed with exit status 1
error: Bad exit status from /var/tmp/rpm-tmp.PjSlGE (%build)
```
Test Plan: make local
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4954802
Signature: t1:4954802:1493224431:55d7123f523c29adb5d0d97d5030fa1c6b0f4845
Summary: As it was noted in D4851138, writing to stdout may break extensions.
Test Plan: arc unit
Reviewers: #mercurial, simpkins
Reviewed By: simpkins
Subscribers: net-systems-diffs@fb.com, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4948924
Tasks: 17453793
Signature: t1:4948924:1493151885:4bcb971e65ac623aa5f91cf80fee5af6902c9226
Summary:
clang is unhappy about implicit converting `const char *` to `const unsigned
char *`. This patch adds explicit casting to fix that.
Test Plan: Run `make local` on an OS X machine.
Reviewers: wez, #sourcecontrol, sid0
Reviewed By: sid0
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4952123
Signature: t1:4952123:1493171270:16fae06afc56d060d7cfd4a845cef6271137f92d
Summary:
The extension was a quick hack from me to make it easier to locate non-core
code when doing extension development. It seems a good idea to add colors in
general so it's named "morecolors" with the hope that we can colorize more
places in the future.
Test Plan: Added a test
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4884999
Signature: t1:4884999:1492473666:89a6756b231eac4274fa05d489480229146bdab8
Summary:
this simplifies the code and resolves a `new[]/delete`
allocator/deallocator mismatch that I tripped over in D4929693
I think this is pretty straight forward. The diff is made a little
larger than I would have liked because there were some missing `const`
qualifiers that had to be propagated out to various callers.
Test Plan:
ran the tests:
```
11:23 $ rt
.................................s.........s.s.s......s..sss............s.....s...................................................................................................................
Skipped test-p4fastimport-import-incremental.t: missing feature: Perforce server and client
Skipped test-p4fastimport-criss-cross.t: missing feature: Perforce server and client
Skipped test-p4fastimport-case-insensitive-rename.t: missing feature: Perforce server and client
Skipped test-p4fastimport-import-modes.t: missing feature: Perforce server and client
Skipped test-p4fastimport-import.t: missing feature: Perforce server and client
Skipped test-p4fastimport-import-deletes.t: missing feature: Perforce server and client
Skipped test-p4fastimport-case-insensitivity.t: missing feature: Perforce server and client
Skipped test-p4fastimport-import-parallel.t: missing feature: Perforce server and client
Skipped test-infinitepush-backup-sql.t: missing getdb.sh
Skipped test-infinitepush-sql.t: missing getdb.sh
# Ran 184 tests, 10 skipped, 0 warned, 0 failed.
```
Reviewers: simpkins, durham
Reviewed By: durham
Subscribers: net-systems-diffs@fb.com, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4930821
Signature: t1:4930821:1493160075:5ee1d452b394c4d6f347029b0ff3fd89377624c5
Summary:
This diff changes our code to use the new SHA1 library. See the previous diff
for why we do this.
Test Plan:
Run related tests manually:
```
$ make local PYTHON=python2
$ rt test-remotefilelog-*.t
.........................
# Ran 25 tests, 0 skipped, 0 warned, 0 failed.
$ rt test-treemanifest*.t
........
# Ran 8 tests, 0 skipped, 0 warned, 0 failed.
$ rt test-fastmanifest*.t
.........
# Ran 9 tests, 0 skipped, 0 warned, 0 failed.
```
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4945025
Signature: t1:4945025:1493154873:844e55a51ab250354fc08163e0949eed47b0a861
Summary:
This imports a SHA1 hash function implementation from
https://github.com/cr-marcstevens/sha1collisiondetection, commit hash
0572d8a302b1f62f25c1559b0de74908550ab2c3, using the MIT License.
This allows us to decouple from an external crypto library like openssl. The
new library's "safe hash" feature could also mitigate known SHA1 collisions.
test-check-code is modified accordingly to skip checking the sha1 code
without modifying them.
Test Plan: Run test-check-code.t and make sure it's not complaining.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: ikostia, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4945012
Signature: t1:4945012:1493154822:be0deb451d50f3a8b6cc42addf091349bf6554f4
Summary:
We now use `check-code.py` directly from core mercurial. It's no longer
necessary to have it in our repo.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, jeroenv, rmcelroy
Reviewed By: jeroenv, rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4945054
Signature: t1:4945054:1493122097:5880dc2c0caac215cb10db40d30691dc1a147315
Summary:
Previously we ignore "draft -> public" phase move, and new nodes returned by
the server may remain "draft" until the next pull. This diff records the
obsmarkers and updates phases of the new nodes correctly so they become
public without an extra hg pull.
Test Plan: Modified an existing testcase
Reviewers: #mercurial, durham, rmcelroy
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4934720
Signature: t1:4934720:1493076241:9b23da1140b68aece9e838c604e64611b3498794
Summary:
This memory is allocated using operator new[] and
must therefore be deleted with delete [] to avoid breaking
the world.
This is D4929693 but applied to the source of truth.
Test Plan:
this resolved a crash in eden and was found using ASAN.
`rt` continues to pass
Reviewers: durham, simpkins
Reviewed By: simpkins
Subscribers: net-systems-diffs@fb.com, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4943100
Signature: t1:4943100:1493146425:2d99dd60efd6a4811312a7eee8f24a76c72dd396
Summary:
This basically takes the last upstream version of my conflictinfo patch and makes it an extension, so we can get it out to FB/Nuclide users
a bit faster. @asriram has already been developing with this version as a personal extension for a couple of weeks.
I still plan to ship the upstream version through after the freeze ends -- when that happens, we can delete this extension. During the time the two
versions overlap, they shouldn't conflict. (The extension version will still run over the internal version until it's disabled, though.)
Review-wise, this is similar to the last upstream version, except it adds the "command" key that indicates which command generated the conflicts (based on which mutually-exclusive state file exists), and incorporates most of the feedback.
Test Plan: Added a test and ran associated tests.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, asriram, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4944709
Signature: t1:4944709:1493148790:a4e798f5bd17ada767ae6c96fe8c8ab973960383
Summary: I was using """_dotfiles_scm_info '[%s] '""" in my prompt, which broke after the refactoring. Add a unit test and fix the code.
Test Plan: wrote a test for the breakage, edited code until they all passed
Reviewers: rmcelroy, tja
Reviewed By: rmcelroy
Subscribers: mjpieters, #pe_security
Differential Revision: https://phabricator.intern.facebook.com/D4945132
Signature: t1:4945132:1493136425:4516151982d173584cf874985de9de1fed0962d5
Summary:
Implement an option to set a bookmark after we imported successfully.
We used to try to calculate this for continuous imports in a wrapper around the
importer. However it's much easier to just do it inside the importer itself, in
particular when we add branch support later
Test Plan: rt test-p4*
Reviewers: #idi, #sourcecontrol, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4935717
Signature: t1:4935717:1493127835:262955b3288d9bd03ca08a45d7ec1667d786430a
Summary:
Use the with statement to lock/unlock. This is more failsafe
and less error prone.
Test Plan: rt test-p4*
Reviewers: #sourcecontrol, #idi, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4921212
Signature: t1:4921212:1492696169:8a7075068252063f140bedeb58ddf70e36a138f4
Summary:
correct PYTHONPATH to allow running tests without having to
source hg-dev. This was suggested by @durham
Test Plan: rt tests-p4*
Reviewers: #sourcecontrol, #idi, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters, durham
Differential Revision: https://phabricator.intern.facebook.com/D4921199
Signature: t1:4921199:1492696261:ca70d837a9f273433b21fb10e3f4ff1afc6a5698
Summary:
Perforce client support a view. A view maps a server side path to a client side
path, e.g.: the view '//depot/A/B/... //myclient/foo/...' maps every file in
//depot/A/B/ on the server side to a local checkout foo/ inside the root for
checkouts defined for the client 'myclient'.
We are using §p4 where§ to support these mappings. We do the mapping inside the
FileImporter at the moment as this runs nicely in parallel. It's a bit hacky
but get's the job done. We use this mostly to ommit the common prefix
//depot/... and remove branch indicators such as Main.
So in our case a view looks like
//depot/Software/OculusSDK/PC/Main... //client/Software/OculusSDK/PC/...
resulting in a file
//depot/Software/OculusSDK/PC/Main/test.txt
being imported as
Software/OculusSDK/PC/test.xt
Test Plan: rt test-p4*
Reviewers: #sourcecontrol, #idi, ikostia
Reviewed By: ikostia
Subscribers: ikostia, durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4913483
Signature: t1:4913483:1492702356:b97b691343b8a1d52940445934730b31d411db4c
Summary:
Similar to LFS, it can be useful to read the metadata about imported
commits after the import. We write the imported p4 changelist <> hg changeset
map to an sqlite database. This will allow wrapper scripts to easily allow
using it without having to traverse the full hg history. In particularly it
allows to check if new revs were imported.
Test Plan: rt test-p4*
Reviewers: #sourcecontrol, #idi, quark
Reviewed By: quark
Subscribers: quark, wlis, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4913476
Signature: t1:4913476:1493127257:b69191de034908f493e0c68fda3b56ff8703e2d4
Summary:
We do not write the blobs to local cache anymore. We want our LFS
server to import them from Perforce directly or serve them from Perforce
directly. In order to do so, we need the correct mapping from oid to perforce
file + cl. This is generally useful metainformation that other LFS
implementation can use. We simple write the data to sqlite because it's simple
and built in.
Test Plan: rt test-p4*
Reviewers: #sourcecontrol, #idi, quark
Reviewed By: quark
Subscribers: quark, durham, wlis, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4913469
Signature: t1:4913469:1492796253:1e3b389c7cb0ba3acf9504410d267a1cf9651118
Summary:
Add a special mode to the importer that patches the LFS extension to
not write blobs to local disc. In our case we do have the files already in
Perforce and do not have to write them again to disk. This is currently breaking
verify and therefore we are patching verify.
Test Plan: rt test-p4*
Reviewers: #sourcecontrol, #idi, durham, quark
Reviewed By: quark
Subscribers: quark, durham, wlis, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4913455
Signature: t1:4913455:1492979419:204c1075376fe975ddea880b22e6984684e7ff25
Summary:
Command that checks if a commit was backed up.
It does it by checking if revisions are in local backup state.
Test Plan: Run unit tests and lints
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4938072
Tasks: 17519836
Signature: t1:4938072:1493072854:291765bb59d327db8504feb47d6089818ae1e11a
Summary: That's an interesting piece of information that we'd like to know.
Test Plan: Enable patched logginghelper.py extension and check that new field is logged
Reviewers: #mercurial, simonfar
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4936862
Tasks: 17453793
Summary:
infinitepushbackup.tempcleanworkingcopiesbackups option sends special command
to clean backup bookmarks from the server for the shared working copy (i.e.
if we have main repo `fbsource` and shared working copy `fbsource2` we want
to have backups only for `fbsource` not for `fbsource` and `fbsource2`).
Before this diff cleanup commands were send for every backup.
This diff makes them send only if backup state file is present.
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4929489
Signature: t1:4929489:1493075761:a408300338a10864043b538540d03880a49c4e1a
This allows us to turn off the obsolescence warning altogether, since we still
use strip together with obsolescence in some situations (e.g. changing the
narrowspec for narrowhg, in which case we "insert" revisions which were
previously not relevant in the middle of the change log and then have to
strip and re-create their children to repoint p1).
Summary:
There was a recent change in mercurial and revsets
shouldn't return lists anymore. This diff fixes it.
This diff also fixes git diff handling. It returned not even a list,
but a single rev number. Unfortunately it's not tested because we need to
mock conduit response.
Test Plan: arc unit
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: ikostia, kulshrax, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4938930
Signature: t1:4938930:1493051696:cfb278773e5932d5f38942b0a0ca49f018ec5083
Summary:
This diff changes lfs `revision(raw=False)` output to include hg filelog
metadata. The LFS blob does not contain filelog metadata as before.
This hurts performance if there is a rename, or the binary starts with the
magic `\1\n`. But compatibility is greatly improved - it's now possible to swap
a non-lfs revision with mercurial rename to a lfs revision, and easier to be
compatible with remotefilelog (namely, remotefilelog defers filelog.add until
commit hash is known).
Test Plan: Modified existing test.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: rmcelroy, durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4906074
Signature: t1:4906074:1492559513:09b25fc1026d4ce8fd784a044d6724f12e8bda45
Summary:
@sid0 noticed absorb invalidating fsmonitorstate, causing `hg status` to be
slow. Unfortunately, `dirstate.rebuild` seems to be the only way to move
working directory parent without changing working copy, which is also used
by `hg reset` and removing state invalidation will cause `test-reset.t` to
fail. Fortunately, in absorb's case, it's fine to not invalidate fsmonitorstate
because no new status will occur outside ctx.files (`hg status` after absorb is
a subset of `hg status` before absorb).
Test Plan: `rt --extra-config-opt=extensions.fsmonitor= test-absorb*.t --chg`
Reviewers: #mercurial, sid0
Reviewed By: sid0
Subscribers: mjpieters, sid0
Differential Revision: https://phabricator.intern.facebook.com/D4925426
Signature: t1:4925426:1492796143:0d6cd4fd46ce130286d0b6ea5a01eadf350397b8
Summary:
There was a bug that caused commits with file deletions to not be backed up.
This diff fixes it by first checking if file exists in the commit and only
then downloading context of the file.
Note:
In tests I had to ignore stdout of `hg pushbackup` because the output was
different on macs and linux.
On linux there was an additional line
remote: abort: data/committostripfirst.i@091b63e5e4: no match found!
Probably mac's remotefilelog closes stdout/stderr earlier, but I wasn't
able to find a root cause.
Test Plan: Run unit tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4929244
Tasks: 15389402
Signature: t1:4929244:1492791804:77b2baa9eb54a53120a955e72e6c132be5db6b44
Summary:
Upstream Mercurial has enabled python deprecation warning in the tests, so
python 2.6 runs now complain about our use of context.nested(). The current fix
of marking all those functions with suppression decorators breaks the tests in
some environments since the suppression module can't be found unless it's
installed or the environment has been set up correctly.
Let's just get rid of our use of nested() and the nodeprecate module.
Test Plan: Ran the tests on centos7 with python 2.7
Reviewers: wez, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4920337
Signature: t1:4920337:1492676584:1c26f2673c14e79793a3ee80c3281fd1bc76a313
Summary:
`exchange.readbundle()` can return bundle2 unbundler or changegroup cg1unpacker.
In case of cg1unpacker let's just read it from the stream
Test Plan: arc unit
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4913313
Tasks: 15389402
Signature: t1:4913313:1492623174:72abe4d1e449ae31d78a6c98b554c0406e8a2ea2
Summary: Unused import breaks tests. Can be added later when it is needed.
Test Plan: - rt
Reviewers: davidsp, #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4921372
Summary:
BSD and GNU ls behave differently in case of a missing dir:
- exit code is [1] vs [2]
- message is also slightly different
We want this to be unified for both platforms.
Test Plan:
- run the test on Linux
- hope for the best on OSX
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4921369
Summary:
Implement incremental imports.
1. Find the last imported perforce changelist.
2. Set startctx and use it in all importers
3. Import filelogs from their current position (we "should" add an additional check here, but we don't)
4. Import manifests and changelists.
Manifests are a bit tricky because we must obtain the original filelog revision
*BEFORE* we imported them, but manifest imports come "after". We could read the
most recent entry from manifests, but that won't cover the case in which files
are added. So instead we know the changelist that we are currently importing,
and looking for the rev with the correct linkrev in filelogs. That's a big ugly,
but it works. We could instead return the original offset from the worker and
pass it into the manifest importer, but I feel that is not much better and
evenutally more errorp rone.
Test Plan: cd tests && rt test-p4*
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #idi
Differential Revision: https://phabricator.intern.facebook.com/D4890110
Signature: t1:4890110:1492662991:0e141e62734e1224ac8e1c11f4e8794452455b18
Summary: remove the unused copytracing code until we use it again.
Test Plan: rt test-p4*
Reviewers: #sourcecontrol, #idi, wlis
Reviewed By: wlis
Subscribers: wlis, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4913446
Signature: t1:4913446:1492624958:f6c083f3c64352f0a8e1172a1d6c6338ee86cd24
Summary:
This makes it more clear we are passing changelist numbers and not Mercurial
revs.
Test Plan: cd tests && rt test-p4*
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters, #idi
Differential Revision: https://phabricator.intern.facebook.com/D4890097
Signature: t1:4890097:1492474348:a788fd96a066bede986e693271bd773e33d19048
Summary:
More detailed debug output for manifests. For changelog we don't
use it at the moment as changelog imports uses the perforce date, causing
changelog hashes not being stable. Will fix this in the future.
Test Plan: cd tests && rt test-p4*
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #idi
Differential Revision: https://phabricator.intern.facebook.com/D4890093
Signature: t1:4890093:1492474213:2b410da985fdd5d0786ef9dde05ecbb96f157e14
Summary:
Use repo lookup to get p1 and p2 and use their manifestnodes instead.
We will use this for incremental imports in order to correctly optain the
manifestnode.
Test Plan: cd tests && rt test-p4*
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #idi
Differential Revision: https://phabricator.intern.facebook.com/D4890084
Signature: t1:4890084:1492474086:91658b5c57b4e58af88ee14673ffa3516bc2d88a
Summary:
Return the filename from the worker so that we can later use it
for better messages and pass it correctly to progress().
Test Plan: cd tests && rt test-p4*
Reviewers: #sourcecontrol, durham, ikostia
Reviewed By: ikostia
Subscribers: ikostia, durham, mjpieters, #idi
Differential Revision: https://phabricator.intern.facebook.com/D4890079
Signature: t1:4890079:1492643347:10eb254ba99faf9e28107ede3ef78f1fcab7946a
Summary:
Large repacks had some long pauses where there was no feedback to the user. This
adds more granualar progress updates.
Test Plan: Ran repack in a large repo and saw more granular progress
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4915492
Signature: t1:4915492:1492641504:9db31d534fe201bec838e77ba470c9051d3be04f
