Commit Graph

2720 Commits

Author SHA1 Message Date
David Soria Parra
f8bb4e2122 lfs: add a 'null' store that doesn't require any setup
Summary:
We have to use the dummy store to ensure we don't try to open an HTTP
connection during push, which can fail and must not happen during p4fastimport.
However `dummy` requires some configuration. So just add an internal `null`
store that allows us to operate LFS in a mode that will ignore all large files
completely.

Test Plan: used it in p4fastimport

Reviewers: #sourcecontrol, quark

Reviewed By: quark

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4963638

Signature: t1:4963638:1493316573:c012bd97794f9a57c3cf8c15d868a67ae3c03c31
2017-04-27 11:13:54 -07:00
Durham Goode
2b12f514c7 treemanifest: include tree pack during pushes
Summary:
To enable pushing between peers (and eventually pushing to the server), let's
teach bundle creation to include the trees being pushed.

Test Plan: Adds a test

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4957456

Signature: t1:4957456:1493266296:67f98a2b3d691644bde9098a713d05266f349cde
2017-04-27 10:44:34 -07:00
Durham Goode
afacc9433b treemanifest: unify client and server tree history access patterns
Summary:
Previously the server would access the tree data in an adhoc manner. Sometimes
it would talk straight to revlogs, sometimes it would create stores and talk to
data packs. This patch makes it access trees the same way clients do, through
repo.svfs.manifestdatastore and manifesthistorystore.

This also cleans up the client store creation just a little and adds a
unionmetadatastore for unified history access.

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4957441

Signature: t1:4957441:1493263349:e76d177f7a9f45343e6f984d6c0ae2c7cacba035
2017-04-27 10:44:34 -07:00
Durham Goode
2f2f03d8d5 store: add history.getnodeinfo api
Summary:
Previously, history stores only had getancestors() apis which returned all the
ancestors. This was expensive if there was a lot of ancestors, like for the root
tree of treemanifests. Let's add an api for accessing a single history entry.
This will be useful in pack generation for only fetching the history related to
the trees we're sending at that time.

Test Plan: Added a test

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4957432

Signature: t1:4957432:1493263124:a155ac5a70c35f7e25a5cc48c9d9c2126d4c5858
2017-04-27 10:44:34 -07:00
Durham Goode
2a7e625b34 treemanifest: add history pack data during commit
Summary:
Previously, a local commit would only write data packs, and it just threw away
the history data entirely. Let's add history packs and record them on commit.

Test Plan:
The tests are updated to show these new packs. In some cases the
datapacks got smaller as well, since they can now take advantage of history data
for delta choices.

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4956105

Signature: t1:4956105:1493265399:d3fa1052c207fba0045cbb92b4d833d18d48e099
2017-04-27 10:44:34 -07:00
Durham Goode
df97f609bf treemanifest: move history sorting responsibility into MutableHistoryPack
Summary:
Previously, the logic that added data to a mutable history pack was required to
add it in the correct order (all entries for a certain file at once, and in
newest-first order). This required the callers to jump through weird hoops if
the data came in out of order or at different times in the transaction.

This patch moves the ordering logic to be inside MutableHistoryPack, so callers
can add the data in any order they wish, and it will get sorted before being
serialized.

This does add memory pressure to things that read a lot of history, like repack.
If this becomes a problem we may want to add a 'historypack.flush()' api that
let's us tell the history pack it's ok to flush it's current contents to disk.

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4956096

Signature: t1:4956096:1493264693:a2275a49e35565d4b11244e3e5dd82c25de7e16e
2017-04-27 10:44:34 -07:00
Durham Goode
7e3a739970 treemanifest: set allowincomplete to true for repacks
Summary:
It's possible for history packs to not have all of history, so let's allow that
during repacks.

Also fix one exception to use hex()

Test Plan: A future patch has tests that exposed this

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4956078

Signature: t1:4956078:1493264261:81c12520b7352d5040cdb027509c975678c10069
2017-04-27 10:44:34 -07:00
Durham Goode
59a339f8d5 treemanifest: fix issue with initial local tree creation
Summary:
The code that creates new trees when the user commits only creates new trees if
the parent commit has a tree already. If the commit is the very first commit in
the repo (i.e. p1 == nullid), then the parent tree is empty.  The `if tree:`
check considered this not-a-tree and therefore the first commit did not generate
a tree. Let's switch to `if tree is None:` to handle this.

A future patch adds tests around peer-to-peer pushing/pulling that encounters
this.

Test Plan: Future patch tests caught this

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4956069

Signature: t1:4956069:1493263952:c3d59f3e48dda623fd8c67630cb4bcd474e63518
2017-04-27 10:44:34 -07:00
Durham Goode
df6bf53021 treemanifest: fix bug with remote store
Summary:
The remote store needs to throw a KeyError if it can't find the node. Returning
None was just causing breakages up stream and meant the union store did not move
on to the next store. This was caught by peer-to-peer pushing tests in a future
patch.

Test Plan: Future tests caught this

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4956062

Signature: t1:4956062:1493264179:432a3add7c0501804520e9abc92e33ba8fec4384
2017-04-27 10:44:34 -07:00
Durham Goode
ba9a6b0e28 treemanifest: change generatepackstream api
Summary:
The old func api parsed the getpack arguments internally. Let's move that out to
the wire protocol level, so in future patches we can reuse generatepackstream
more easily.

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4956038

Signature: t1:4956038:1493265461:e7ed1ad659fb0639c325df3f8352dbe06f650740
2017-04-27 10:44:33 -07:00
Durham Goode
198bc173fb treemanifest: move pack wire protocol to operate within bundle2
Summary:
Previously the wire pack protocol was a custom stream of bytes. This
patch moves it to be within bundle2 so we can take advantage of features it has
(versioning, arguments, mid-stream error messaging, including packs in other
bundles, etc).  This will be useful in future patches where we want to allow
peer-to-peer exchange of client only tree manifest data.

This also removes all the custom argument encoding/decoding.

Test Plan:
Ran the tests. Order changed but the content remains the same and the
order is consistent.

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4924718

Signature: t1:4924718:1493050942:2734cfdbca0b0b0a3f3be1ee8e224f634553c305
2017-04-27 10:44:33 -07:00
Durham Goode
2b154613e2 pack: switch readers to read from file handle
Summary:
A future patch will move the pack wireprotocol to use bundle2. In this new world
we'll be given a file handle instead of a remote peer, so let's switch the
utility methods to work on a file handle instead.

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4924553

Signature: t1:4924553:1493050882:7a9ee8b282bf47ef393362dd0114d801dc2a68d5
2017-04-27 10:44:33 -07:00
Durham Goode
7e4775c6ff treemanifest: enable auto prefetching trees during hg pull
Summary:
Adds a config option that will automatically download the last N trees from the
server during pull. It efficiently decides what bases to set, based on what
commit parents it already has.

Test Plan: Added a test

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4923211

Signature: t1:4923211:1493200730:39872db6aabb48c12131ff8501a950c8ee60e2b0
2017-04-27 10:44:33 -07:00
Kostia Balytskyi
b331423be1 phabricator: look for .arcrc in the correct location on Windows
Summary:
On Windows `arc install-certificate` writes the cert to APPDATA:
https://fburl.com/i7fpssf8, so that is where we need to look for it.

Test Plan: - make this change, see that `hg ssl` shows commit status locally

Reviewers: davidsp, andrasbelo, #sourcecontrol

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4962334

Tasks: 17511545
2017-04-27 09:01:15 -07:00
Martijn Pieters
05d97a9461 sparse: provide a change summary on file and rule count deltas
Summary:
This allows a third-party tool like Buck report more detail on what a sparse
command actually achieved.

Test Plan: Run the test suite.

Reviewers: #sourcecontrol, durham, rmcelroy, ikostia

Reviewed By: ikostia

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4962550

Tasks: 17150508

Tags: autosparse

Signature: t1:4962550:1493307403:d14e16d079121688a4eb69f0d69cd02ab55c4936
2017-04-27 08:37:13 -07:00
Jun Wu
0d6151f530 remotefilelog: add lfs integration test
Summary:
The test covers common workflows like clone, commit, push, update, pull. It
exercises the remotefilelog plain store and Python datapack store to make sure
they won't lose the revlog flag. The test also tries to verify rename works
correctly.

Since the lfs extension may be eventually upstreamed, it seems a good idea to
make remotefilelog call `lfs.wrapfilelog` so lfs is free from remotefilelog
code.

Test Plan: Added a test

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4904281

Signature: t1:4904281:1492560308:5fd9f214ada6de795735ea7d737d30c1bf39812a
2017-04-26 19:55:02 -07:00
Jun Wu
2b3b8e46a5 remotefilelog: add filelog methods
Summary:
Filelog methods like `addrevision`, `revision(raw=True)` are needed for flag
processor (lfs) to work correctly. Add them in remotefilelog so lfs wrapper
code could replace them.

Test Plan:
Run existing tests. Stronger tests and lfs integration test will be added when
this area is more complete.

Reviewers: #mercurial, durham, rmcelroy

Reviewed By: durham

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4903959

Signature: t1:4903959:1493163809:5ebd88fac21d8225a12ce68bfc63a2867ee43769
2017-04-26 19:52:20 -07:00
Jun Wu
4240bd017e remotefilelog: let content stores support metadata
Summary:
This diffs add a `getmeta` method to all content stores. The cdatapack code is
modified to pass the tests, it needs further change to support `getmeta`.

The datapack format is bumped to v1 from v0. For v1, we append a `metadata`
dict at the end of each revision. The dict is currently used to store revlog
flags and rawsize of raw revlog fulltext. In the future we can put more data
like a second hash etc, without changing API or format again.

This diff focuses on correctness. A datapack caching layer to speed up
`getmeta` will be added later.

Tests are updated since we write new v1 packfile now and the format change
leads to different content and packfile names.

`Makefile`, `ls-l.py` are added to make tests easier to maintain.

Test Plan: Updated existing tests.

Reviewers: #mercurial, rmcelroy, durham

Reviewed By: durham

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4903917

Signature: t1:4903917:1493255844:7ef5d487096cd2f78f2aaae672a68d49f33632ee
2017-04-26 19:50:36 -07:00
Phil Cohen
6efbf50599 progressfile: remove trailing space
Summary: Today is a bad day. I don't even know how this got in there.

Test Plan: y

Reviewers: durham, #mercurial

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4957784
2017-04-26 14:37:58 -07:00
Phil Cohen
3ee70495aa progressfile: add PYTHONPATH export to test
Summary: Otherwise, the test requires hg-dev and will break third-party developers.

Test Plan: Reran test

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4957662

Signature: t1:4957662:1493242115:7e83b130791b07b1e45f04ce199e5c4ee9d58d77
2017-04-26 14:32:13 -07:00
Phil Cohen
cf88ecb166 progressfile: fix trailing space
Summary:
To my deep shame, I discovered today that `arc land` does not rerun `arc unit` if you introduce post-review changes.

Sorry!

Test Plan: Ran arc unit this time :)

Reviewers: quark, #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4957528

Signature: t1:4957528:1493241560:9208e745ac92c0e771b0a4b4bac91d6c1feed025
2017-04-26 14:19:39 -07:00
Phil Cohen
22969a6bf8 progress: add progress.statefile config
Summary:
allows users to have JSON progress bar information written to a path

Controlled by the `ui.progressfile` config. Mercurial will overwrite this file
each time the progress bar is updated.

The schema of this file is (JSON):

- topics: array of topics from oldest to newest. (last is always the active one)
- state: map of topic names to objects with keys:
    - topic (e.g. "changesets", "manifests")
    - pos: which item number out of <total> we're processing
    - total: total number of items
    - unit: name of the type of unit being processed (e.g., "changeset")
    - item: the active item being processed (e.g., "changeset #5")
    - active: whether this is the currently active progress bar
    - units_per_sec: if active, how many <unit>s per sec we're processing
    - speed_str: if active, a human-readable string of how many <unit>s per sec
        we're processing
    - estimate_sec: an estimate of how much time is left, in seconds
    - estimate_str: if active, a human-readable string estimate of how much time
        is left (e.g. "2m30s")


Test Plan: added a test

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: rmcelroy, quark, stash, asriram, mjpieters, durham

Differential Revision: https://phabricator.intern.facebook.com/D4752788

Signature: t1:4752788:1493233878:b49095237d32233c78cd0e0aaaa7b94e0e9e7011
2017-04-26 14:05:44 -07:00
Jun Wu
8fcd86af16 remotefilelog: move constants to class to prepare index format change
Summary:
To be able to bump version and change formats, the related constants need to
be moved to individual classes. So a class (ex. datapack) can be subclassed to
handle different formats.

Test Plan: `arc unit`

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4927284

Signature: t1:4927284:1493152641:e3274dd735d50baf193b7615dd314f4e6cf161f0
2017-04-26 13:34:15 -07:00
Jun Wu
0e2c18e2cd remotefilelog: add revlog flags information to the protocol
Summary:
Make the unpacked file format to include the revlog flag information, and make
the getfile(s) protocol support it.

Note: The `getpackv1` protocol and packfile format is not changed yet.

Test Plan:
Run existing tests. Stronger tests and lfs integration test will be added when
this area is more complete.

Reviewers: #mercurial, rmcelroy, durham

Reviewed By: durham

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4903772

Signature: t1:4903772:1493152451:ab393b0208f0eee199ffc4c8fcfdfd5dd6d0f3ac
2017-04-26 13:08:13 -07:00
Rodrigo Damazio Bovendorp
a72494903f packaging: adding Cython as a build dependency for the Debian package
Now that absorb is included in the deb, we need Cython to build linelog.
2017-04-26 11:30:42 -07:00
Rodrigo Damazio Bovendorp
930bb6277b packaging: removing libssl dependency from deb package 2017-04-26 11:30:42 -07:00
Wez Furlong
031681855c cstore: centos6 doesn't know about nullptr
Summary: second time's the charm

Test Plan: make local, wait for contbuild

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4955570

Signature: t1:4955570:1493230082:3493ac29ff8d04a98e50f83febc9dd827e9b4a7c
2017-04-26 11:09:59 -07:00
Wez Furlong
539ac2a3d0 cstore: fixup missing nullptr declaration
Summary:
this is failing the contbuild:

```
In file included from cstore/pythonutil.h:23,
                 from cstore/py-datapackstore.h:21,
                 from cstore/py-cstore.cpp:17:
cstore/../cstore/store.h: In member function  const char* ConstantStringRef::content() :
cstore/../cstore/store.h:37: error:  nullptr  was not declared in this scope
error: command 'gcc' failed with exit status 1
error: Bad exit status from /var/tmp/rpm-tmp.PjSlGE (%build)
```

Test Plan: make local

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4954802

Signature: t1:4954802:1493224431:55d7123f523c29adb5d0d97d5030fa1c6b0f4845
2017-04-26 09:35:31 -07:00
Stanislau Hlebik
64ecdf2691 fastpartialmatch: handle nullid correctly
Summary: Special case for nullid

Test Plan: arc unit

Reviewers: #mercurial, simpkins, ikostia

Reviewed By: ikostia

Subscribers: net-systems-diffs@fb.com, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4945927

Tasks: 17622253

Signature: t1:4945927:1493123209:712990c1e883241d99c0aeb4af8d48e18e1baa3d
2017-04-26 03:55:59 -07:00
Stanislau Hlebik
c48413d98b cleanobsstore: write cleanobsstore output to stderr
Summary: As it was noted in D4851138, writing to stdout may break extensions.

Test Plan: arc unit

Reviewers: #mercurial, simpkins

Reviewed By: simpkins

Subscribers: net-systems-diffs@fb.com, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4948924

Tasks: 17453793

Signature: t1:4948924:1493151885:4bcb971e65ac623aa5f91cf80fee5af6902c9226
2017-04-26 03:55:16 -07:00
Jun Wu
fc3ca42ee0 sha1: fix complication error
Summary:
clang is unhappy about implicit converting `const char *` to `const unsigned
char *`.  This patch adds explicit casting to fix that.

Test Plan: Run `make local` on an OS X machine.

Reviewers: wez, #sourcecontrol, sid0

Reviewed By: sid0

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4952123

Signature: t1:4952123:1493171270:16fae06afc56d060d7cfd4a845cef6271137f92d
2017-04-25 20:21:14 -07:00
Jun Wu
6c6ebce1f6 morecolors: new extension to add more colors to outputs
Summary:
The extension was a quick hack from me to make it easier to locate non-core
code when doing extension development. It seems a good idea to add colors in
general so it's named "morecolors" with the hope that we can colorize more
places in the future.

Test Plan: Added a test

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4884999

Signature: t1:4884999:1492473666:89a6756b231eac4274fa05d489480229146bdab8
2017-04-25 17:03:26 -07:00
Wez Furlong
f02ec48128 treemanifest: ConstantStringRef -> shared_ptr<string>
Summary:
this simplifies the code and resolves a `new[]/delete`
allocator/deallocator mismatch that I tripped over in D4929693

I think this is pretty straight forward.  The diff is made a little
larger than I would have liked because there were some missing `const`
qualifiers that had to be propagated out to various callers.

Test Plan:
ran the tests:

```
11:23 $ rt
.................................s.........s.s.s......s..sss............s.....s...................................................................................................................
Skipped test-p4fastimport-import-incremental.t: missing feature: Perforce server and client
Skipped test-p4fastimport-criss-cross.t: missing feature: Perforce server and client
Skipped test-p4fastimport-case-insensitive-rename.t: missing feature: Perforce server and client
Skipped test-p4fastimport-import-modes.t: missing feature: Perforce server and client
Skipped test-p4fastimport-import.t: missing feature: Perforce server and client
Skipped test-p4fastimport-import-deletes.t: missing feature: Perforce server and client
Skipped test-p4fastimport-case-insensitivity.t: missing feature: Perforce server and client
Skipped test-p4fastimport-import-parallel.t: missing feature: Perforce server and client
Skipped test-infinitepush-backup-sql.t: missing getdb.sh
Skipped test-infinitepush-sql.t: missing getdb.sh
# Ran 184 tests, 10 skipped, 0 warned, 0 failed.
```

Reviewers: simpkins, durham

Reviewed By: durham

Subscribers: net-systems-diffs@fb.com, mjpieters, #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D4930821

Signature: t1:4930821:1493160075:5ee1d452b394c4d6f347029b0ff3fd89377624c5
2017-04-25 15:44:55 -07:00
Jun Wu
80e40da9d6 sha1: switch to new implementation
Summary:
This diff changes our code to use the new SHA1 library. See the previous diff
for why we do this.

Test Plan:
Run related tests manually:

```
$ make local PYTHON=python2
$ rt test-remotefilelog-*.t
.........................
# Ran 25 tests, 0 skipped, 0 warned, 0 failed.
$ rt test-treemanifest*.t
........
# Ran 8 tests, 0 skipped, 0 warned, 0 failed.
$ rt test-fastmanifest*.t
.........
# Ran 9 tests, 0 skipped, 0 warned, 0 failed.
```

Reviewers: #sourcecontrol, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4945025

Signature: t1:4945025:1493154873:844e55a51ab250354fc08163e0949eed47b0a861
2017-04-25 14:53:32 -07:00
Jun Wu
76c7347a2e clib: import sha1 impl from github/cr-marcstevens/sha1collisiondetection
Summary:
This imports a SHA1 hash function implementation from
https://github.com/cr-marcstevens/sha1collisiondetection, commit hash
0572d8a302b1f62f25c1559b0de74908550ab2c3, using the MIT License.

This allows us to decouple from an external crypto library like openssl. The
new library's "safe hash" feature could also mitigate known SHA1 collisions.

test-check-code is modified accordingly to skip checking the sha1 code
without modifying them.

Test Plan: Run test-check-code.t and make sure it's not complaining.

Reviewers: #sourcecontrol, durham

Reviewed By: durham

Subscribers: ikostia, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4945012

Signature: t1:4945012:1493154822:be0deb451d50f3a8b6cc42addf091349bf6554f4
2017-04-25 14:53:04 -07:00
Jun Wu
bedfb2642c check-code: remove check-code.py
Summary:
We now use `check-code.py` directly from core mercurial. It's no longer
necessary to have it in our repo.

Test Plan: `arc unit`

Reviewers: #sourcecontrol, jeroenv, rmcelroy

Reviewed By: jeroenv, rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4945054

Signature: t1:4945054:1493122097:5880dc2c0caac215cb10db40d30691dc1a147315
2017-04-25 14:47:17 -07:00
Jun Wu
44c4a80415 pushrebase: deal with phase move correctly
Summary:
Previously we ignore "draft -> public" phase move, and new nodes returned by
the server may remain "draft" until the next pull. This diff records the
obsmarkers and updates phases of the new nodes correctly so they become
public without an extra hg pull.

Test Plan: Modified an existing testcase

Reviewers: #mercurial, durham, rmcelroy

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4934720

Signature: t1:4934720:1493076241:9b23da1140b68aece9e838c604e64611b3498794
2017-04-25 14:46:36 -07:00
Wez Furlong
446905c07d cstore: fix heap corruption
Summary:
This memory is allocated using operator new[] and
must therefore be deleted with delete [] to avoid breaking
the world.

This is D4929693 but applied to the source of truth.

Test Plan:
this resolved a crash in eden and was found using ASAN.

`rt` continues to pass

Reviewers: durham, simpkins

Reviewed By: simpkins

Subscribers: net-systems-diffs@fb.com, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4943100

Signature: t1:4943100:1493146425:2d99dd60efd6a4811312a7eee8f24a76c72dd396
2017-04-25 13:34:01 -07:00
Phil Cohen
eb2814985f resolve: add internal:dumpjson as an internal merge tool (as an extension, for now)
Summary:
This basically takes the last upstream version of my conflictinfo patch and makes it an extension, so we can get it out to FB/Nuclide users
a bit faster. @asriram has already been developing with this version as a personal extension for a couple of weeks.

I still plan to ship the upstream version through after the freeze ends -- when that happens, we can delete this extension. During the time the two
versions overlap, they shouldn't conflict. (The extension version will still run over the internal version until it's disabled, though.)

Review-wise, this is similar to the last upstream version, except it adds the "command" key that indicates which command generated the conflicts (based on which mutually-exclusive state file exists), and incorporates most of the feedback.

Test Plan: Added a test and ran associated tests.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, asriram, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4944709

Signature: t1:4944709:1493148790:a4e798f5bd17ada767ae6c96fe8c8ab973960383
2017-04-25 12:46:28 -07:00
Greg Prosser
094b505444 scm-prompt: fix _dotfiles_scm_info
Summary: I was using """_dotfiles_scm_info '[%s] '""" in my prompt, which broke after the refactoring.  Add a unit test and fix the code.

Test Plan: wrote a test for the breakage, edited code until they all passed

Reviewers: rmcelroy, tja

Reviewed By: rmcelroy

Subscribers: mjpieters, #pe_security

Differential Revision: https://phabricator.intern.facebook.com/D4945132

Signature: t1:4945132:1493136425:4516151982d173584cf874985de9de1fed0962d5
2017-04-25 10:56:35 -07:00
David Soria Parra
63370254c8 p4fastimport: implement --bookmark
Summary:
Implement an option to set a bookmark after we imported successfully.
We used to try to calculate this for continuous imports in a wrapper around the
importer. However it's much easier to just do it inside the importer itself, in
particular when we add branch support later

Test Plan: rt test-p4*

Reviewers: #idi, #sourcecontrol, quark

Reviewed By: quark

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4935717

Signature: t1:4935717:1493127835:262955b3288d9bd03ca08a45d7ec1667d786430a
2017-04-25 15:29:39 +01:00
David Soria Parra
4d406cac74 p4fastimport: use with statement
Summary:
Use the with statement to lock/unlock. This is more failsafe
and less error prone.

Test Plan: rt test-p4*

Reviewers: #sourcecontrol, #idi, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4921212

Signature: t1:4921212:1492696169:8a7075068252063f140bedeb58ddf70e36a138f4
2017-04-25 15:29:39 +01:00
David Soria Parra
757087ede8 p4fastimport: clarify criss-cross comment
Summary: Clarify the criss-cross comment and our monotonic property.

Test Plan: None

Reviewers: #sourcecontrol, #idi, ikostia

Reviewed By: ikostia

Subscribers: ikostia, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4921201

Signature: t1:4921201:1492696996:c2c8a179926649fac5009388d3c3054a627a9e10
2017-04-25 15:29:39 +01:00
David Soria Parra
0337eae344 p4fastimport: correct PYTHONPATH for tests
Summary:
correct PYTHONPATH to allow running tests without having to
source hg-dev. This was suggested by @durham

Test Plan: rt tests-p4*

Reviewers: #sourcecontrol, #idi, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters, durham

Differential Revision: https://phabricator.intern.facebook.com/D4921199

Signature: t1:4921199:1492696261:ca70d837a9f273433b21fb10e3f4ff1afc6a5698
2017-04-25 15:29:39 +01:00
David Soria Parra
78dcff9f00 p4fastimport: support client view
Summary:
Perforce client support a view. A view maps a server side path to a client side
path, e.g.: the view '//depot/A/B/... //myclient/foo/...' maps every file in
//depot/A/B/ on the server side to a local checkout foo/ inside the root for
checkouts defined for the client 'myclient'.
We are using §p4 where§ to support these mappings. We do the mapping inside the
FileImporter at the moment as this runs nicely in parallel. It's a bit hacky
but get's the job done. We use this mostly to ommit the common prefix
//depot/... and remove branch indicators such as Main.

So in our case a view looks like

  //depot/Software/OculusSDK/PC/Main... //client/Software/OculusSDK/PC/...

resulting in a file

  //depot/Software/OculusSDK/PC/Main/test.txt

being imported as

  Software/OculusSDK/PC/test.xt

Test Plan: rt test-p4*

Reviewers: #sourcecontrol, #idi, ikostia

Reviewed By: ikostia

Subscribers: ikostia, durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4913483

Signature: t1:4913483:1492702356:b97b691343b8a1d52940445934730b31d411db4c
2017-04-25 15:29:39 +01:00
David Soria Parra
3799abce68 p4fastimport: write metadata to sqlite
Summary:
Similar to LFS, it can be useful to read the metadata about imported
commits after the import. We write the imported p4 changelist <> hg changeset
map to an sqlite database. This will allow wrapper scripts to easily allow
using it without having to traverse the full hg history. In particularly it
allows to check if new revs were imported.

Test Plan: rt test-p4*

Reviewers: #sourcecontrol, #idi, quark

Reviewed By: quark

Subscribers: quark, wlis, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4913476

Signature: t1:4913476:1493127257:b69191de034908f493e0c68fda3b56ff8703e2d4
2017-04-25 15:29:39 +01:00
David Soria Parra
2ab5a0baf8 p4fastimport: support for writing LFS metadata to sqlite
Summary:
We do not write the blobs to local cache anymore. We want our LFS
server to import them from Perforce directly or serve them from Perforce
directly. In order to do so, we need the correct mapping from oid to perforce
file + cl. This is generally useful metainformation that other LFS
implementation can use. We simple write the data to sqlite because it's simple
and built in.

Test Plan: rt test-p4*

Reviewers: #sourcecontrol, #idi, quark

Reviewed By: quark

Subscribers: quark, durham, wlis, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4913469

Signature: t1:4913469:1492796253:1e3b389c7cb0ba3acf9504410d267a1cf9651118
2017-04-25 15:29:39 +01:00
David Soria Parra
7d78fa39bf p4fastimport: initial support for writing lfs metadata
Summary:
Add a special mode to the importer that patches the LFS extension to
not write blobs to local disc. In our case we do have the files already in
Perforce and do not have to write them again to disk. This is currently breaking
verify and therefore we are patching verify.

Test Plan: rt test-p4*

Reviewers: #sourcecontrol, #idi, durham, quark

Reviewed By: quark

Subscribers: quark, durham, wlis, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4913455

Signature: t1:4913455:1492979419:204c1075376fe975ddea880b22e6984684e7ff25
2017-04-25 15:29:39 +01:00
Stanislau Hlebik
1642b009fa infinitepush: add isbackedup command
Summary:
Command that checks if a commit was backed up.
It does it by checking if revisions are in local backup state.

Test Plan: Run unit tests and lints

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4938072

Tasks: 17519836

Signature: t1:4938072:1493072854:291765bb59d327db8504feb47d6089818ae1e11a
2017-04-25 01:09:28 -07:00
Stanislau Hlebik
8595e6160c logginghelper: log obsstore size
Summary: That's an interesting piece of information that we'd like to know.

Test Plan: Enable patched logginghelper.py extension and check that new field is logged

Reviewers: #mercurial, simonfar

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4936862

Tasks: 17453793
2017-04-25 01:09:28 -07:00