Summary:
Backport the split command and its dependencies (namely, `bookmarksupdater`)
from mutable-history changeset cb0d62ed5e2a using GPL2 license.
There are some adjustments to make the ported code work well with existing
code, namely:
- The `--norebase` flag was renamed as `--no-rebase` and moved to the
backported command so we no longer wrap the split command.
- Rebase now runs inside a same transaction.
The glob change in tests is because `(tip)` shows up.
Test Plan: arc unit
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: ikostia, mjpieters, medson
Differential Revision: https://phabricator.intern.facebook.com/D5254544
Signature: t1:5254544:1497519470:fe8a7308b35b578fd4f1257cebba15062e752c2f
Summary:
This diff removes "unset HGRCPATH" to try to run tests in a more
controlled environment. And skip the test if system HG looks broken.
Test Plan: arc unit
Reviewers: simpkins, #mercurial, stash
Reviewed By: stash
Subscribers: net-systems-diffs@fb.com, mjpieters, medson
Differential Revision: https://phabricator.intern.facebook.com/D5263484
Signature: t1:5263484:1497604378:28ec94c33ba77680c8d2325a8fa36457e0eb8c91
Summary:
This removes Cython as a build dependency as requested by our users.
Added a test to prevent check-in files with `/home/`.
Modified check-code to skip checking Cython generated files.
Test Plan:
Run `make clean local` with Cython installed and uninstalled.
Also run `make clean local` with `USECYTHON=1` and `0`.
Reviewers: #mercurial, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters, medson
Differential Revision: https://phabricator.intern.facebook.com/D5252935
Signature: t1:5252935:1497525190:402798077d44e39e24fcb037535ec7ffd1af9c4b
Summary:
Special command that fills optional metadata about infinitepush nodes. For now
it's info about changed files + info about how many lines were
added/removed. Since information is saved in json format it's easily
extensible.
It's a separate command so that we will be able to run it asynchronously:
during infinitepush push separate `hg debugfillinfinitepushmetadata` process
will be started. Push process won't wait for `debugfillinfinitepushmetadata`
process.
sqlindexapi will be implemented in the next diffs.
Test Plan: arc unit
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters, medson, azich, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5237547
Tasks: 19103180
Signature: t1:5237547:1497397211:569d1ffe3be37c8d14fb7e07e42cd085ad5686d4
Summary:
Previously infinitepush lookup function didn't call original lookup function.
It made it impossible to work with extension like gitlookup that also
overwrite lookup. This diff fixes it.
Test Plan: arc unit
Reviewers: #mercurial
Subscribers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5246683
Tasks: 17733144
Summary:
Let's add a special field that would store arbitrary metadata in json format.
It can be used to store, for example, lists of changes files, file diffs etc.
Depends on D5236375
Test Plan: arc unit
Reviewers: #mercurial, azich
Subscribers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5236394
Tasks: 19103180
Summary:
People want update to optionally not activate a bookmark.
This diff adds an `--inactive` flag to do that.
Test Plan: Added test-update-i.t
Reviewers: durham, #mercurial, quark
Reviewed By: quark
Subscribers: mitrandir, mjpieters, medson, durham, akushner, quark
Differential Revision: https://phabricator.intern.facebook.com/D5228552
Tasks: 8097957
Signature: t1:5228552:1497396566:16322e1a2dfdab30a213ddc37cbde4fc71bffcc0
Before this patch, the pretxnchangegroup 'node' argument would be the last
added node whereas mercurial documention specifies that it should be the first
new changeset. The 'node_last' argument has also been added to both
pretxnchangegroup and changegroup hooks as per mercurial documentation.
The 'node' argument points to the last new changeset. Mercurial documentation
state that it should be the first new changeset.
This will be fixed with the following changeset
Summary:
Previously, if a tree needed to be downloaded on demand (like during hg
show), it would download the entire tree. With this patch, we look backwards in
the changelog for a tree we already have and then download only the new parts of
the tree.
Test Plan:
The existing demanddownload test changed because of this. I've added
more output to the test to show that only new items are downloaded (to be
specific subdir/ is not downloaded again since it did not change).
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters, medson
Differential Revision: https://phabricator.intern.facebook.com/D5210268
Signature: t1:5210268:1497299426:195517184d53396849d0ba8c75216d07741e5863
Summary:
Move movement commands (next, previous) to `movement.py`. Some common utilities
were moved to `common.py`. I did some minor cleanups on the help output.
Test Plan: arc unit
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: mjpieters, medson
Differential Revision: https://phabricator.intern.facebook.com/D5222981
Signature: t1:5222981:1497265815:5dbba53fe7a16dbf924898331496a5c50b54b9f2
Summary: `fbamend.py` is already long. It will become more complex so let's move it.
Test Plan: arc unit
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Differential Revision: https://phabricator.intern.facebook.com/D5209475
Signature: t1:5209475:1497260957:e84aec4a2f90c8dd1d247b2b0caef48cfbb59c55
This helps doing some validation before accepting a pushrebase.
For example, if you want to prevent users to accidentally pushrebase
changesets to the wrong branch by checking if all pushed changesets are
on the same branch as the destination (--to)
Summary:
There was a change in core mercurial b319e3173a9534a2b7750b69ef446a803c3145ff
that catches all exception from uisetup and extsetup. Let's throw subclass of
BaseException in this case.
Test Plan: arc unit
Reviewers: #mercurial, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters, medson, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5236650
Signature: t1:5236650:1497363414:9bbde3cabb696bb77a7ffddcb576923e5743c993
There was a bug that was introduced upstream in 469914605447 and fixed in
758d59a5f3515b18c767af69f4ed28060f8af56c. This diff reflects the changes.
Also filestat was changed in 4091c920ac07e27ee8536715297127e56d536dab.
Summary:
Upstream mercurial --stat option shows only aggregated statistics for all
files. This diff make it possible to get diff statistics per file.
It can be used by automation.
Test Plan: arc unit
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters, medson, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5203835
Tasks: 18508761
Signature: t1:5203835:1496981602:c8a649158892d05442d8d0aa717589974b92ff6b
Summary:
There is a problem with socket.gethostname() because it can return
inconsistent names. See attached task for details.
Instead this diff lets us specify any hostname.
Depends on D5180633
Test Plan: arc unit
Reviewers: #mercurial, simpkins, tja
Subscribers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5180665
Tasks: 18528165
Summary:
Let's add an option that can be used to trigger full clean backup.
It can be useful in many situations, for example, in case of bug in
infinitepush backup or if the naming scheme of backup bookmarks were changed.
Test Plan: arc unit
Reviewers: #mercurial
Subscribers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5180390
Tasks: 15389402
Summary:
We do not want to switch people to obsshelve and then find out that their
traditional shelves cannot be unshelved.
Test Plan: - check that added test passes
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5180998
Summary: Just for an early detection of bad config being distributed.
Test Plan: - add a test case
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5180986
Summary:
See bottom commit of the stack for the motivation.
This commit copies obsshelve-related tests from hotfixed core and cleans them
up a little bit.
Test Plan: - run just added test
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5180981
Summary:
With upstream bab25235cfb5daf578a598bd7ae08f7a70ef7acd `hg strip` also strips
markers too. This diff just accepts the test changes since they seems not dangerous
Test Plan: arc unit
Reviewers: #mercurial, kulshrax, simpkins, quark
Reviewed By: quark
Subscribers: net-systems-diffs@fb.com, mjpieters, medson, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5185690
Signature: t1:5185690:1496700479:a230a2861c8d3cafda3867c81b8fc90a7cfcc4df
Summary:
Previously, we would only look at the node parents to determine what the base
node would be for a prefetch. This didn't work so well when we started
prefetching arbitrary revs, like only prefetching master. This patch allows us
to scan the changelog back a bit to find the nearest base tree.
Test Plan: Added a test
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5108686
Signature: t1:5108686:1495565589:1729bfae7198e8db2223446b81e3bdbf77cf3db6
Summary:
This adds a config option that allows clients to prefetch only certain trees
after a pull. This is useful if the user wants to download just master. A future
patch will make it only download the portion of the master tree that is new.
Test Plan: Added a test
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5108677
Signature: t1:5108677:1495565526:a443c41ccbf55e529c93aa0c021849396d611ac6
After upstream commit bab25235cfb5daf578a598bd7ae08f7a70ef7acd strip now also
deletes obsmarkers that are related to the stripped commits and saves them in
the bundle.
Summary:
Simple copy tracking for draft commits. It doesn't use db to save move info
because getting move info is quite fast already.
Depends on D5137372
Test Plan: Run tests
Reviewers: #mercurial, durham, quark, rmcelroy
Reviewed By: quark
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5137886
Tasks: 18508761
Signature: t1:5137886:1496348869:8f4a9761fdeb26ddee5e18a318cc85582ef4adbb
Summary:
Copytracing that is based on a simple idea: most moves are either directory
moves or moves of the file inside the same directory. That means that either
basename of the moved file or the dirname of the moved file is the same.
More details in the comments.
Test Plan: Run unit-tests
Reviewers: #mercurial, durham, quark, rmcelroy
Reviewed By: quark, rmcelroy
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5137372
Tasks: 18508761
Signature: t1:5137372:1496243148:8d229c1593da196b674318ee8b37af15a60831c8
Summary:
Fix the tweakdefaults code to extract the --tool argument when extracting
--rebase, and to pass it in to the rebasemodule.rebase() function correctly.
Test Plan: Included a new test.
Reviewers: #mercurial, rmcelroy, quark
Reviewed By: quark
Subscribers: quark, net-systems-diffs@fb.com, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5134466
Tasks: 18770938
Signature: t1:5134466:1495848690:308a8d3fbad39e6ecfeaee04b1b3267c1fa1118c
Summary:
This diff makes the tests use the vendored directaccess and inhibit instead the
foreign ones.
Test Plan: arc unit
Reviewers: #mercurial, sid0
Reviewed By: sid0
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5142981
Signature: t1:5142981:1496173851:3a8f3a1c689711dbef517c893bf72c90ab144761
Summary: Fix lint errors and make the test pass.
Test Plan: arc unit
Reviewers: #sourcecontrol, sid0
Reviewed By: sid0
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5142477
Signature: t1:5142477:1496173824:bf941d895d3259fc788c09d4fefca7dddd11a33b
Summary:
Vendored using GPL2 license from https://bitbucket.org/marmoute/mutable-history
changeset c9f1118b33d60f8faa4b89988a8155c544f5bb0d without modification.
Test Plan: N/A. The code was imported as is.
Reviewers: #sourcecontrol, sid0
Reviewed By: sid0
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5142063
Signature: t1:5142063:1496173803:d0a9fa50a8423a531432ac3cff378b50952684b1
Summary:
copytracing extension is going to be simpler than I expected in the beginning.
There is no need to separate folder, let's move it to hgext3rd
Test Plan: arc unit
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5146585
Tasks: 18508761
Signature: t1:5146585:1496147733:ad6ad14f663a8b9bf4b687f0767395321f9840b7
Summary:
Previously `_getrevstobackup()` would load manifest for each commit in memory
only to check if a file was deleted in this commit or not. Manifest can be
quite big and since every loaded flatmanifest is cached in fastmanifest then
memory usage can be huge.
Since we are doing this only to check if there are any commits that were
downloaded to the client without filelogs and then were stripped server-side.
This cases are rare and it would be easier to track these commits in the
config file.
Test Plan: arc unit
Reviewers: #mercurial, mjpieters, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5111372
Tasks: 15389402
Signature: t1:5111372:1495848867:4028bd48313ac0e2022c3695195bedc6eda80abf
Summary:
`revlog.ancestor` is expensive on long changelog, but is only called a few
(about 30) times for smartlog usage. Therefore we could simply cache the
result in a key-value database.
This speeds up smartlog by about 200ms.
Test Plan: Added a new test
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5135746
Signature: t1:5135746:1495783503:411260551fd29fda42c9fc809b56f8f77fb2eaf4
Summary:
There are two `convert.h`. This patch unifies them and does cleanups so it's
a valid header file which could be included by multiple .c and .cpp files
and linker won't complain re-definition (by adding `static` to everything).
Besides, reformat the code so it could pass check-code. Also fix a compiler
warning about comparing an unsigned integer with a signed integer.
Test Plan: `make clean local`. It still builds.
Reviewers: durham, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5113550
Signature: t1:5113550:1495565413:e399f898ac513e64af37dab5daf55cedbabfc703
Summary: This is what extensions should belong to.
Test Plan: Changed existing test.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5113042
Signature: t1:5113042:1495561987:d1af0f7cebfdb8b0c1a6bb71cdc826d817a4e441
Summary:
We now use setup.py to do the build so this is outdated. It does not seem to be
useful so just remove it.
Test Plan: eyes
Reviewers: durham, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, ttung, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5112853
Signature: t1:5112853:1495561149:a4ee408e1fe06dea32f96ae93abb38d7d21d3619
Summary:
statprof is now part of core hg. And core hg will import its own version
instead of the external one. So it's no longer to have a separate version.
I did a comparison. The version in fb-hgext is falling behind - it lacks of
Chrome support. The upstream version is also cleaner in terms of Python3 code
standard and error handling.
Test Plan: eyes
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5112666
Signature: t1:5112666:1495562274:7e1ca7d964bf617695dbefd59c3843a3ebf47f9e
Summary:
Let's log whenever one merge parent was deleted and another was modified.
It will help us find out how much disabled copytracing affects users.
Test Plan: test-copytrace-errormsg.t
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D5103311
Tasks: 18508761
Summary:
First step to get better copytracing. For now let's just change merge error
message to suggest re-running command with enabled copytracing.
Test Plan: arc unit
Reviewers: #mercurial, rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5102174
Tasks: 18508761
Summary:
Wrap the capabilities wire protocol command to print the hostname to stderr.
This commit contains the implementation of the extension but not code to register it on a particular repo.
Test Plan: `rt`
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: durham, wez, stash, simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5087634
Tasks: 16758829
Signature: t1:5087634:1495498748:213a32c5c474145371bbbc43b0463dee31868a9a
Summary: Upstream has deprecated cmdutil.commands() in favor of registrar.commands()
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5106486
Signature: t1:5106486:1495485074:0e20f00622cc651e8c9dda837f84dd84cc51099e
Summary:
Previously, the progress bar is file-level - it moves when a file is transferred.
When uploading or downloading a single giant file, progress bar matters. And
this diff adds it to make people more patient.
Test Plan:
Manually upload and download several big files.
Make sure the progress bar appears in both cases.
Also make sure the output is sane with `-v`.
Reviewers: #mercurial, davidsp, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5100014
Signature: t1:5100014:1495316719:c500ccd63d3a495e41575cadd72419a41f5fb259
Summary:
The `cmp` logic does not look special from the original `basefilectx.cmp` so it
could be removed. This means we will benefit from LFS's override of
`basefilectx.cmp` and have a fast path for LFS binary diff.
Test Plan: Added a test. Make sure the old code fails that test.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5100529
Signature: t1:5100529:1495468055:ff7fa30f3277b942bf49ba1e9b8764effacd972a
Summary:
Previously our data packs would have incredibly long delta chains, where to read
the last entry in the file you had to read every previous delta all the way to
the beginning. This was very expensive for chains of 2+ million deltas. Let's
add a config option to limit how long the deltas get, and for now we'll default
it to 1000.
Test Plan: Added a test
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5095603
Signature: t1:5095603:1495213707:737d63129cf459ad6927a1f4deb0dda5a8ce0a7f
Summary:
The newly added config option could skip LFS revision checks. Let's use it
so `hg verify` passes.
Test Plan: arc unit
Reviewers: #mercurial, davidsp
Reviewed By: davidsp
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5091709
Signature: t1:5091709:1495163508:3f36dfa221ab0896e1dc49d30c762d7957ac9826
Summary:
This allows automation to know which modified or added files are LFS for
specific changesets.
Test Plan: Added a test
Reviewers: #mercurial, davidsp
Reviewed By: davidsp
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5091591
Signature: t1:5091591:1495163606:138638d8ccc57b8ed6c1e324750ec5dc15c198a0
This test had non-deterministic ordering, which caused it to fail sometimes.
Let's make the test check more explicitly, by checking that the bad commit hash
is not present.
Summary:
One of the heads may point to filtered commit and isbackedup command fails in
this case. Let's use repo.unfiltered()
Test Plan: arc unit
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5001977
Signature: t1:5001977:1493907725:f6a138bd4e2cae48b64152fceaf73660ea91f43b
Summary:
`repair.strip` removes required revisions and their children unconditionally.
That means innocent children may be removed. This patch fixes it.
We may want to have an in-core function for this feature to not strip innocent
children.
Thanks Zibi Braniecki from Mozilla for reporting [1]!
[1]: https://bitbucket.org/facebook/hg-experimental/issues/6/hg-absorb-merges-diverged-commits
Test Plan: Added a test
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5073342
Signature: t1:5073342:1494972231:ca2a9881415119047771c8f35e059f4bfb1749e5
Summary:
When a bundle is created, the bundle could be exchanged via copy and possibly
eventually reach to a publishing repo. If we don't upload LFS blobs, hg server
could have revisions that can never be checked out or verified.
So let's just assume bundles generated by `hg bundle` will be public, and
upload LFS blobs automatically, without depending on other code review tooling.
Note: there is a `preoutgoing` hook which will be triggered in this case,
however it's not useful since it does not have the `outgoing` information.
Test Plan: Will add a new test
Reviewers: davidsp, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5050945
Signature: t1:5050945:1494958648:0c46ab1d85755838ba189bbb1e0673882922bb58
Summary:
The `filectx.cmp` fast path allows us to show "binary file changed" diff output
when two binaries have the same size but different content.
Test Plan: Added a test case
Reviewers: davidsp, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5067266
Signature: t1:5067266:1494954783:a41e8213782afe24618ef2930c9576f21610fd3e
Summary:
When the server performs a repack, it would read all the data from all the
revlogs. This was very slow and expensive. Let's add a --incremental option that
makes it only read the revlog entries that have a linkrev newer than the latest
linkrev data that's already in a pack file.
Test Plan: Adds a test
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4997260
Signature: t1:4997260:1493904216:c4f5b6c9652bbf8f66c1bc2b2547c898324d22cd
Summary:
Calculating what blobs to upload should be fast since we use changelog `files`
information. Users will see `lfs: uploading ...` very soon. So let's downgrade
`lfs: computing set of blobs to upload` to a debug message to make `-v`
cleaner.
Test Plan: Updated existing test.
Reviewers: davidsp, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5066600
Signature: t1:5066600:1494954115:c00925a1930f9b53e914078d32b6c4e4161099ee
Summary:
Downloading or uploading a single LFS object could take long. Instead of
showing a summary of what objects are downloaded or uploaded, print a message
per object if --verbose is provided.
Test Plan: Added a test
Reviewers: davidsp, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5066327
Signature: t1:5066327:1494953894:3593436ff53eaff117a2932476dfc4b867fb9f17
Summary:
Previously, when an object is not found server-side, we error out during the
basic transfer API. This diffs move the check to the batch metadata API so the
check is preformed earlier.
That also means, when batch fetching many LFS objects, we could report multiple
objects being missing instead of just one.
Test Plan: Modified an existing test
Reviewers: davidsp, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5065623
Signature: t1:5065623:1494952131:cb1df73e1cea21e07bfd0ec630bacec60e5b385c
Summary:
We are adding a simple transaction that works only with filelogs but
allows for concurrent access from multiple workers. This allows for a proper
rollback in case of a failure in a worker process, which previously would result
in bad data in the repositoriy.
Test Plan: rt test-p4* test-check*
Reviewers: #mercurial, durham, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5070340
Signature: t1:5070340:1494958313:b10b1eac5b42b36d1a587c4ae1c95fc2f8b5ad35
Summary:
The LFS uploading or downloading process consists of 2 parts: a batch API to
get metadata about objects, followed by `len(objects)` requests.
This diff moves the first metadata API out from a giant `_batch` method to
make the code structure easier to maintain. Later diffs will further split
`_batch` method.
A side effect is `lfs: mapping blobs to #{action} URLs` message is removed.
More user-friendly message will be added back in a later patch.
Reviewers: #mercurial, davidsp, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5057541
Signature: t1:5057541:1494949989:d510c3d9369744e7ee776c0d0a26d263dfaf8352
Summary:
Displaying total bytes to upload is currently inaccurate - the server could
already have some objects so only a subset of selected objects will be
uploaded.
Besides, we pass `pointers` to related upload and download APIs so `total`
could be calculated from them.
This diff removes the inaccurate "need to upload" message and unnecessary
parameters. An accurate message will be added in a later patch.
Reviewers: #mercurial, davidsp, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5057470
Signature: t1:5057470:1494949554:07e637a8b58c894598b3f0c2dd53f80a5ade25a8
Summary:
Upstream shelve now writes simplekeyvaluefile for shelvedstate. This means
that tests which relied on ordered state files need to be adjusted.
Test Plan: - rt
Reviewers: durham, mitrandir, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D5069982
Summary: Added -U/--unified NUM to fbshow.py, and used that to set diff.unified=NUM.
Test Plan: hg show --help; hg show -U 1; hg show -U 2; hg show -U by induction.
Reviewers: mburman, simonfar, ikostia
Differential Revision: https://phabricator.intern.facebook.com/D5052156
Tasks: 10006036
Tags: python
Summary:
The server or the HTTP library may provide more detailed error message.
Let's use them. `RequestFailedError` is also renamed to `LfsRemoteError` to
make it clear it's related to LFS and network.
Test Plan: arc unit
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5057014
Signature: t1:5057014:1494634052:00c1cb1b337a0e4cb92828cd49e95b7b0ff7e540
Summary:
move p4fastimport under hgext3rd as that's where it belongs.
Also has the benefit that we package it up.
Test Plan: rt test-p4*
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5056664
Signature: t1:5056664:1494631758:4892f47922d8fbbd2c1f7793f3f29ee73fa8fa42
Summary:
Read the Git-LFS specification [1] and implement most checks to prevent
programming error and detect data corruption.
The new code should be stronger than what was before the refactoring, since the
old code only checks keys but not values.
[1]: https://github.com/git-lfs/git-lfs/blob/master/docs/spec.md
Test Plan: Added a new test
Reviewers: #mercurial, davidsp, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5046740
Signature: t1:5046740:1494543072:2f0dbfc07cd52a10572550a277d850bdf3f78d27
Summary:
This diff adds a shared `p4setup.sh` that de-duplicates common logic among
tests. It also uses absolute path to make sure the extension being tested is
the version being developed.
The LFS test is also workarounded temporarily waiting for upstream change.
Test Plan: Run existing tests
Reviewers: #mercurial, davidsp
Reviewed By: davidsp
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5049279
Signature: t1:5049279:1494547832:28222fd2034115faca73860d6dd2f19206701aaa
Facebook probably doesn't want external users reaching out to them directly
for support, so this should be customizable to specify other possible courses
of action (especially in cases where githelp is further extended to document
git-related company-internal commands in other companies).
It looks like there is some flakeyness in racing pushes during this test. Seems
to be caused by hg serve loading the initial repo state while other processes
are writing new commits, which results in a missing master. Introducing a slight
sleep to the beginning of the transaction allows all the processes to load the
correct initial state and gets rid of the flakeyness.
Summary:
This hooks into the strip code to allow stripping revisions from treemanifests
as well. This will make it easier to clean up server repos when mistakes happen.
Test Plan: Adds a test
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D5031489
Summary:
Previously, when creating a new manifestrevlog for the trees, we attempted to
copy the opener options and insert the treemanifest value for the manifestrevlog
class to pick up. It turns out that the opener we copied was actually just a
wrapper around the same underlying object that the real repo used, so when we
did opener.options = oldopener.options.copy(), we were actually putting the new
copy on both the old and new opener. This meant we enabled treemanifest for the
entire repo.
This wasn't a problem in most cases because the manifestlog had already been
loaded. But in a pushrebase world, all the data structures are refreshed when
the lock is obtained (after waiting for the other push to finish). In this
situation, the refreshed repo.manifestlog was now a treemanifest so the second
commit was pushed to the main manifest as a tree.
The fix is to not use the opener.options hack to configure the tree. Instead we
are refactoring upstream to take an explicit options to the manifestrevlog
constructor. This patch makes our internal treemanifest extension use this new
api and adds a test for push contention.
Test Plan: Added a test. It failed before, and passes after.
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D5026019
Summary:
`filelog.filenode` could be `None` when it's unknown - like working copy. That
breaks `isbinary` test. Let's modify `_islfs` check to return False to disable
lfs code path in that case.
Test Plan: Added a new test.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5034058
Signature: t1:5034058:1494431790:a02b69addf5d2f9b6b9a27ad71ed4f136b2bfd2b
Summary: `revdiff` should use raw revisions. 5d11b5ed in core hg is a similar fix.
Test Plan: Added a test.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5031916
Signature: t1:5031916:1494366439:80d496ed6f69a784d345510147eab6c479496f84
Summary:
77b66b03be changed how pushrebase and obsmarkers worked, which apparently causes
this test to change. Updating the test.
Test Plan: ran it
Reviewers: quark
Reviewed By: quark
Differential Revision: https://phabricator.intern.facebook.com/D5030830
Signature: t1:5030830:1494360593:e0c9ce9a8466cb725b3dcc22a9161c9f02b00f15
Summary:
The config option was designed to be used server-side to avoid accessing remote
LFS blob stores. However, we now have `lfs.url=null:` as a clean alternative,
which will explicitly raise if the server ever tries to download remote
content. So `bypass` is no longer needed.
`bypass` could also be used for displaying raw content. That is doable using
`debugdata`.
Test Plan: Modified existing tests
Reviewers: #mercurial, rmcelroy, davidsp
Reviewed By: davidsp
Subscribers: davidsp, rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5009725
Signature: t1:5009725:1494260384:332d7bd658359c004342efd01e4f13b6fc5499b1
Summary:
`lfs.blobstore` is a path used to store local blobs. Some of the blobs are not
uploaded yet so they couldn't be easily discarded. Although blobs downloaded
from the server could be removed if necessary.
It does not make much sense to make this a config option - the data should be
stored reliably. It's also dangerous to allow write to arbitrary paths under
`repo.vfs`.
This diff makes the local blob store fixed path at `.hg/store/lfs/objects`.
The choice is similar to `.git/lfs/objects`, but with `store`, shared repo
could share their lfs stores.
Test Plan: Modified existing test.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5009664
Signature: t1:5009664:1493978440:c3f6351d0ea0cca2ce3caa9f7260c5d65bcc0e5f
Summary:
lfs-test-server is a reference implementation by GitHub [1]. Testing against it
will give us more confidence.
[1]: https://github.com/git-lfs/lfs-test-server
Test Plan: Added a test
Reviewers: #mercurial, davidsp
Reviewed By: davidsp
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5009346
Signature: t1:5009346:1494042196:86e8c2e0e5c4296b1afc72efa84a9c1b9030e6fc
Summary:
Per discussion with @davidsp, we want to stick to Git-LFS specification and
avoid non-standard behavior. The chunking behavior will happen at LFS server
transparently.
The direct motivation for this is to make it possible to implement an
efficient `filectx.cmp` that just compares hashes.
Test Plan: Updated existing test
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters, davidsp
Differential Revision: https://phabricator.intern.facebook.com/D5001827
Signature: t1:5001827:1493914639:c58694873e79a8ca910bb8ee01bf593885896664
Summary:
Usually LFS files are binary files. But there could be exceptions. This diff
adds a new customized field `x-is-binary` to record those exceptions.
The `filectx.isbinary` API is changed to use that metadata as a fast path.
This allows us to provide a transparent user experience (whether a file is
stored in LFS or not does not affect its original `isbinary` property),
while still being able to skip loading the LFS blob if the file is binary.
Test Plan: Added a new test case
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5001117
Signature: t1:5001117:1493892406:2a8ebd42d0ae0dbae39b87b9ea16db22b99f3d12
Summary: Like _config, _import needs to restore the pre-existing profile if refreshing failed. Otherwise we have marked new rules as imported and applying without that actually being true.
Test Plan: rt test-sparse*
Reviewers: #sourcecontrol, rmcelroy, durham, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4985415
Tasks: 17793415
Tags: autosparse
Signature: t1:4985415:1493893462:ee4bcbf558706c3d2f58367dd7bf2533ccf1b824
Summary:
D4865150 and D4934720 aren't effective in our current setup. The direct
cause in the code is because the server couldn't find common marker version:
```
# old server-side code, returns empty in our current setup
obsolete.commonversion(bundle2.obsmarkersversion(reply.capabilities))
```
Upon investigation, it's because there is no `exchange` enabled client-side.
But we do want one-way (server->client) markers for the rebased commits, as
long as obsstore is enabled (createmarkers is set, without exchange).
The upstream expects the server to have obsstore enabled, and exchange
enabled, to send markers. Since we are generating markers without an
obsstore (see D4865150), we are on our own way. This diff makes it one step
further.
This diff adds an explicit parameter to the `b2x:rebase` part to tell the
server what obsmarker format the client supports so the server could make a
right decision without relying on the "standard" `reply.capabilities`, which
is affected by the exchange option.
Test Plan: Change the existing test, make sure the old code fails.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4997972
Signature: t1:4997972:1493848751:14c29654b2e8246bd12a8de8820af5b3773e2fb7
The main rationale for this change is that I've got years of use
patterns on bisect where I *start* bisect by running 'hg bisect
--reset', not *finish* with that command, so `hg status` with
morestatus enabled is spewing annoyances in nearly every repository
I've ever used. It wasn't any harder to allow people to ignore
anything they want, but (at least for now) bisect is the only state
that seems "special" in that it doesn't have a definite "end" that
Mercurial can detect and automatically clean up. I'd also be fine with
adding something like morestatus.ignorebisect if we want to avoid a
potential footgun where a user could ignore rebase state.
Summary:
The tests were still testing version 0. Let's update it to version 1 and fix the
bad version check test.
Test Plan: Ran the tests
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4991404
Signature: t1:4991404:1493802873:def4135219d731f7c0cb542012e439e79192c54f
Summary:
Previously looking up a particular node in a histpack required a bisect to find
the file section, then a linear scan to find the particular node. If you needed
to look up the latest 3000 nodes one by one, that involved 3000 linear scans,
many of which traversed the same nodes over and over.
This patch adds additional index at the end of the current histidx file. In a
future patch, we will change getnodeinfo() to use this index instead of the
linear scan logic.
Test Plan:
Ran the tests. I haven't actually verified that the data in these
indexes is correct. My next patch will add logic that reads these indexes and
will add tests around it. I won't land this until I've confirmed it's correct.
Reviewers: quark, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4983690
Signature: t1:4983690:1493798877:ae0802b4896b54bf066df9f684d94554855fd35a
Summary:
Previously, we used the length of the index file to determine the upper bounds
of the bisect. In a future patch we'll want to add more data to the end of the
index file, so we need to record how long the index portion of the index is.
This patch adds that information.
Test Plan: Ran the tests.
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4983682
Signature: t1:4983682:1493693255:57ab9af2030847fedff05b6755113ba8ce0c933b
Summary:
This patch just bumps the histpack version number to 1 and adds a config flag to
enable writing v1 pack files. The format hasn't actually changed in this patch,
I'm just doing the verison bump so I can update all the hashes in the tests
without working about functionality change.
In the next patch I will modify the index format, which won't affect the hashes.
Test Plan:
Ran the tests. I also ran the tests with some debug code to manually
force the sha to include 0 instead of 1 and verified that the hash didn't change
(which confirms that all of these hash changes are just because of that one byte
version change).
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4983675
Signature: t1:4983675:1493692444:5d88df4d46ce487f1b791417754ba000ecf10a1e
Summary:
The newly changed `LZ4_decompress_safe` is unhappy about empty buffer. So let's
check empty revision explicitly.
Test Plan: Added a test
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4986277
Signature: t1:4986277:1493743735:be2ad6047bb0b983425c6e49b7c5ddf2c94d1c1a
Summary:
This test won't be run unless it is in the right directory.
#m brownpaperbag
Test Plan: run arc unit
Reviewers: #sourcecontrol, jeroenv
Reviewed By: jeroenv
Subscribers: jeroenv, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4985738
Tasks: 17796162
Signature: t1:4985738:1493736355:4c9358fbf536661dc6b1f11c546332cda54f21b2
Summary:
Support the action archive. 'archive' means that a revision was
"archived" to a different depot. We must ensure we support the action correctly
in order to have a smooth import.
Test Plan: run it && rt test-p4* test-check*
Reviewers: #sourcecontrol, #idi, wlis
Reviewed By: wlis
Subscribers: wlis, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4980571
Signature: t1:4980571:1493676115:ad0d3748c52747aeb6427fd25ece9d4987886936
Summary:
When passing --limit we are processing N Perforce changelists at a time.
The goal is to provide savepoints for large imports.
Test Plan: rt test-p4* test-check*
Reviewers: #sourcecontrol, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4980482
Signature: t1:4980482:1493688071:800a0bafda33a17cb2ef54c9f399db7055a8cbf9
Summary:
This diff makes 2 changes to v1 packfile metadata:
1. Move `key` in a metadata entry to before `size`.
```
old: [entry-size: 2 byte] [key: 1 byte] [data: var length]
new: [key: 1 byte] [data-size: 2 byte] [data: var length]
```
Previously `entry-size == 0` does not make sense.
2. Use binary to represent sizes, instead of ASCII.
Related utility methods are cleaned up a bit so it's harder to make mistakes.
Test Plan: Updated existing tests
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4983189
Signature: t1:4983189:1493689852:22d544d73ed63fac83f849786de035af304161ce
Summary:
This shows how much overhead the `getmeta` call introduces.
Initial investigation shows the C version is about 10x as fast as than `get` or `getdeltachain`.
Test Plan: Run `hg perfdatapack` in an repo.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4909157
Signature: t1:4909157:1492667211:e82a14e12d04f60c0cee4124e83845ec166a78cc
Summary:
This diff implements getmeta in C and enables related tests.
Now all content stores support `getmeta`.
Test Plan: Run existing tests.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4960926
Signature: t1:4960926:1493611048:55095c32927fac74e698f21d47173cb8a7523fb6
The bundle2 part was always being generated, even for non-treemanifest repos
(because fastmanifest imports the treemanifest module, so the b2partsgenerator
gets registered). This cause pushes to break in any repo with fastmanifest
enabled.
Summary: We've made changes to arcanist configuration recently. Need to do the same in the tests.
Test Plan: arc unit
Reviewers: #scmquery, rmcelroy, ikostia
Reviewed By: ikostia
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4969670
Signature: t1:4969670:1493376937:536470162c52c8be953c813c1fea54f3e506f1e9
Summary:
To enable pushing between peers (and eventually pushing to the server), let's
teach bundle creation to include the trees being pushed.
Test Plan: Adds a test
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4957456
Signature: t1:4957456:1493266296:67f98a2b3d691644bde9098a713d05266f349cde
Summary:
Previously, history stores only had getancestors() apis which returned all the
ancestors. This was expensive if there was a lot of ancestors, like for the root
tree of treemanifests. Let's add an api for accessing a single history entry.
This will be useful in pack generation for only fetching the history related to
the trees we're sending at that time.
Test Plan: Added a test
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4957432
Signature: t1:4957432:1493263124:a155ac5a70c35f7e25a5cc48c9d9c2126d4c5858
Summary:
Previously, a local commit would only write data packs, and it just threw away
the history data entirely. Let's add history packs and record them on commit.
Test Plan:
The tests are updated to show these new packs. In some cases the
datapacks got smaller as well, since they can now take advantage of history data
for delta choices.
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4956105
Signature: t1:4956105:1493265399:d3fa1052c207fba0045cbb92b4d833d18d48e099
Summary:
Previously, the logic that added data to a mutable history pack was required to
add it in the correct order (all entries for a certain file at once, and in
newest-first order). This required the callers to jump through weird hoops if
the data came in out of order or at different times in the transaction.
This patch moves the ordering logic to be inside MutableHistoryPack, so callers
can add the data in any order they wish, and it will get sorted before being
serialized.
This does add memory pressure to things that read a lot of history, like repack.
If this becomes a problem we may want to add a 'historypack.flush()' api that
let's us tell the history pack it's ok to flush it's current contents to disk.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4956096
Signature: t1:4956096:1493264693:a2275a49e35565d4b11244e3e5dd82c25de7e16e
Summary:
Adds a config option that will automatically download the last N trees from the
server during pull. It efficiently decides what bases to set, based on what
commit parents it already has.
Test Plan: Added a test
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4923211
Signature: t1:4923211:1493200730:39872db6aabb48c12131ff8501a950c8ee60e2b0
Summary:
This allows a third-party tool like Buck report more detail on what a sparse
command actually achieved.
Test Plan: Run the test suite.
Reviewers: #sourcecontrol, durham, rmcelroy, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4962550
Tasks: 17150508
Tags: autosparse
Signature: t1:4962550:1493307403:d14e16d079121688a4eb69f0d69cd02ab55c4936
Summary:
The test covers common workflows like clone, commit, push, update, pull. It
exercises the remotefilelog plain store and Python datapack store to make sure
they won't lose the revlog flag. The test also tries to verify rename works
correctly.
Since the lfs extension may be eventually upstreamed, it seems a good idea to
make remotefilelog call `lfs.wrapfilelog` so lfs is free from remotefilelog
code.
Test Plan: Added a test
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4904281
Signature: t1:4904281:1492560308:5fd9f214ada6de795735ea7d737d30c1bf39812a
Summary:
This diffs add a `getmeta` method to all content stores. The cdatapack code is
modified to pass the tests, it needs further change to support `getmeta`.
The datapack format is bumped to v1 from v0. For v1, we append a `metadata`
dict at the end of each revision. The dict is currently used to store revlog
flags and rawsize of raw revlog fulltext. In the future we can put more data
like a second hash etc, without changing API or format again.
This diff focuses on correctness. A datapack caching layer to speed up
`getmeta` will be added later.
Tests are updated since we write new v1 packfile now and the format change
leads to different content and packfile names.
`Makefile`, `ls-l.py` are added to make tests easier to maintain.
Test Plan: Updated existing tests.
Reviewers: #mercurial, rmcelroy, durham
Reviewed By: durham
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4903917
Signature: t1:4903917:1493255844:7ef5d487096cd2f78f2aaae672a68d49f33632ee
Summary: Today is a bad day. I don't even know how this got in there.
Test Plan: y
Reviewers: durham, #mercurial
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4957784
Summary: Otherwise, the test requires hg-dev and will break third-party developers.
Test Plan: Reran test
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4957662
Signature: t1:4957662:1493242115:7e83b130791b07b1e45f04ce199e5c4ee9d58d77
Summary:
allows users to have JSON progress bar information written to a path
Controlled by the `ui.progressfile` config. Mercurial will overwrite this file
each time the progress bar is updated.
The schema of this file is (JSON):
- topics: array of topics from oldest to newest. (last is always the active one)
- state: map of topic names to objects with keys:
- topic (e.g. "changesets", "manifests")
- pos: which item number out of <total> we're processing
- total: total number of items
- unit: name of the type of unit being processed (e.g., "changeset")
- item: the active item being processed (e.g., "changeset #5")
- active: whether this is the currently active progress bar
- units_per_sec: if active, how many <unit>s per sec we're processing
- speed_str: if active, a human-readable string of how many <unit>s per sec
we're processing
- estimate_sec: an estimate of how much time is left, in seconds
- estimate_str: if active, a human-readable string estimate of how much time
is left (e.g. "2m30s")
Test Plan: added a test
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: rmcelroy, quark, stash, asriram, mjpieters, durham
Differential Revision: https://phabricator.intern.facebook.com/D4752788
Signature: t1:4752788:1493233878:b49095237d32233c78cd0e0aaaa7b94e0e9e7011
Summary:
Make the unpacked file format to include the revlog flag information, and make
the getfile(s) protocol support it.
Note: The `getpackv1` protocol and packfile format is not changed yet.
Test Plan:
Run existing tests. Stronger tests and lfs integration test will be added when
this area is more complete.
Reviewers: #mercurial, rmcelroy, durham
Reviewed By: durham
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4903772
Signature: t1:4903772:1493152451:ab393b0208f0eee199ffc4c8fcfdfd5dd6d0f3ac
Summary:
The extension was a quick hack from me to make it easier to locate non-core
code when doing extension development. It seems a good idea to add colors in
general so it's named "morecolors" with the hope that we can colorize more
places in the future.
Test Plan: Added a test
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4884999
Signature: t1:4884999:1492473666:89a6756b231eac4274fa05d489480229146bdab8
Summary:
This imports a SHA1 hash function implementation from
https://github.com/cr-marcstevens/sha1collisiondetection, commit hash
0572d8a302b1f62f25c1559b0de74908550ab2c3, using the MIT License.
This allows us to decouple from an external crypto library like openssl. The
new library's "safe hash" feature could also mitigate known SHA1 collisions.
test-check-code is modified accordingly to skip checking the sha1 code
without modifying them.
Test Plan: Run test-check-code.t and make sure it's not complaining.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: ikostia, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4945012
Signature: t1:4945012:1493154822:be0deb451d50f3a8b6cc42addf091349bf6554f4
Summary:
We now use `check-code.py` directly from core mercurial. It's no longer
necessary to have it in our repo.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, jeroenv, rmcelroy
Reviewed By: jeroenv, rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4945054
Signature: t1:4945054:1493122097:5880dc2c0caac215cb10db40d30691dc1a147315
Summary:
Previously we ignore "draft -> public" phase move, and new nodes returned by
the server may remain "draft" until the next pull. This diff records the
obsmarkers and updates phases of the new nodes correctly so they become
public without an extra hg pull.
Test Plan: Modified an existing testcase
Reviewers: #mercurial, durham, rmcelroy
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4934720
Signature: t1:4934720:1493076241:9b23da1140b68aece9e838c604e64611b3498794
Summary:
This basically takes the last upstream version of my conflictinfo patch and makes it an extension, so we can get it out to FB/Nuclide users
a bit faster. @asriram has already been developing with this version as a personal extension for a couple of weeks.
I still plan to ship the upstream version through after the freeze ends -- when that happens, we can delete this extension. During the time the two
versions overlap, they shouldn't conflict. (The extension version will still run over the internal version until it's disabled, though.)
Review-wise, this is similar to the last upstream version, except it adds the "command" key that indicates which command generated the conflicts (based on which mutually-exclusive state file exists), and incorporates most of the feedback.
Test Plan: Added a test and ran associated tests.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, asriram, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4944709
Signature: t1:4944709:1493148790:a4e798f5bd17ada767ae6c96fe8c8ab973960383
Summary: I was using """_dotfiles_scm_info '[%s] '""" in my prompt, which broke after the refactoring. Add a unit test and fix the code.
Test Plan: wrote a test for the breakage, edited code until they all passed
Reviewers: rmcelroy, tja
Reviewed By: rmcelroy
Subscribers: mjpieters, #pe_security
Differential Revision: https://phabricator.intern.facebook.com/D4945132
Signature: t1:4945132:1493136425:4516151982d173584cf874985de9de1fed0962d5
Summary:
Implement an option to set a bookmark after we imported successfully.
We used to try to calculate this for continuous imports in a wrapper around the
importer. However it's much easier to just do it inside the importer itself, in
particular when we add branch support later
Test Plan: rt test-p4*
Reviewers: #idi, #sourcecontrol, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4935717
Signature: t1:4935717:1493127835:262955b3288d9bd03ca08a45d7ec1667d786430a
Summary:
correct PYTHONPATH to allow running tests without having to
source hg-dev. This was suggested by @durham
Test Plan: rt tests-p4*
Reviewers: #sourcecontrol, #idi, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters, durham
Differential Revision: https://phabricator.intern.facebook.com/D4921199
Signature: t1:4921199:1492696261:ca70d837a9f273433b21fb10e3f4ff1afc6a5698
Summary:
Perforce client support a view. A view maps a server side path to a client side
path, e.g.: the view '//depot/A/B/... //myclient/foo/...' maps every file in
//depot/A/B/ on the server side to a local checkout foo/ inside the root for
checkouts defined for the client 'myclient'.
We are using §p4 where§ to support these mappings. We do the mapping inside the
FileImporter at the moment as this runs nicely in parallel. It's a bit hacky
but get's the job done. We use this mostly to ommit the common prefix
//depot/... and remove branch indicators such as Main.
So in our case a view looks like
//depot/Software/OculusSDK/PC/Main... //client/Software/OculusSDK/PC/...
resulting in a file
//depot/Software/OculusSDK/PC/Main/test.txt
being imported as
Software/OculusSDK/PC/test.xt
Test Plan: rt test-p4*
Reviewers: #sourcecontrol, #idi, ikostia
Reviewed By: ikostia
Subscribers: ikostia, durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4913483
Signature: t1:4913483:1492702356:b97b691343b8a1d52940445934730b31d411db4c
Summary:
We do not write the blobs to local cache anymore. We want our LFS
server to import them from Perforce directly or serve them from Perforce
directly. In order to do so, we need the correct mapping from oid to perforce
file + cl. This is generally useful metainformation that other LFS
implementation can use. We simple write the data to sqlite because it's simple
and built in.
Test Plan: rt test-p4*
Reviewers: #sourcecontrol, #idi, quark
Reviewed By: quark
Subscribers: quark, durham, wlis, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4913469
Signature: t1:4913469:1492796253:1e3b389c7cb0ba3acf9504410d267a1cf9651118
Summary:
Add a special mode to the importer that patches the LFS extension to
not write blobs to local disc. In our case we do have the files already in
Perforce and do not have to write them again to disk. This is currently breaking
verify and therefore we are patching verify.
Test Plan: rt test-p4*
Reviewers: #sourcecontrol, #idi, durham, quark
Reviewed By: quark
Subscribers: quark, durham, wlis, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4913455
Signature: t1:4913455:1492979419:204c1075376fe975ddea880b22e6984684e7ff25
Summary:
Command that checks if a commit was backed up.
It does it by checking if revisions are in local backup state.
Test Plan: Run unit tests and lints
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4938072
Tasks: 17519836
Signature: t1:4938072:1493072854:291765bb59d327db8504feb47d6089818ae1e11a
Summary:
infinitepushbackup.tempcleanworkingcopiesbackups option sends special command
to clean backup bookmarks from the server for the shared working copy (i.e.
if we have main repo `fbsource` and shared working copy `fbsource2` we want
to have backups only for `fbsource` not for `fbsource` and `fbsource2`).
Before this diff cleanup commands were send for every backup.
This diff makes them send only if backup state file is present.
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4929489
Signature: t1:4929489:1493075761:a408300338a10864043b538540d03880a49c4e1a
This allows us to turn off the obsolescence warning altogether, since we still
use strip together with obsolescence in some situations (e.g. changing the
narrowspec for narrowhg, in which case we "insert" revisions which were
previously not relevant in the middle of the change log and then have to
strip and re-create their children to repoint p1).
Summary:
There was a recent change in mercurial and revsets
shouldn't return lists anymore. This diff fixes it.
This diff also fixes git diff handling. It returned not even a list,
but a single rev number. Unfortunately it's not tested because we need to
mock conduit response.
Test Plan: arc unit
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: ikostia, kulshrax, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4938930
Signature: t1:4938930:1493051696:cfb278773e5932d5f38942b0a0ca49f018ec5083
Summary:
This diff changes lfs `revision(raw=False)` output to include hg filelog
metadata. The LFS blob does not contain filelog metadata as before.
This hurts performance if there is a rename, or the binary starts with the
magic `\1\n`. But compatibility is greatly improved - it's now possible to swap
a non-lfs revision with mercurial rename to a lfs revision, and easier to be
compatible with remotefilelog (namely, remotefilelog defers filelog.add until
commit hash is known).
Test Plan: Modified existing test.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: rmcelroy, durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4906074
Signature: t1:4906074:1492559513:09b25fc1026d4ce8fd784a044d6724f12e8bda45
Summary:
There was a bug that caused commits with file deletions to not be backed up.
This diff fixes it by first checking if file exists in the commit and only
then downloading context of the file.
Note:
In tests I had to ignore stdout of `hg pushbackup` because the output was
different on macs and linux.
On linux there was an additional line
remote: abort: data/committostripfirst.i@091b63e5e4: no match found!
Probably mac's remotefilelog closes stdout/stderr earlier, but I wasn't
able to find a root cause.
Test Plan: Run unit tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4929244
Tasks: 15389402
Signature: t1:4929244:1492791804:77b2baa9eb54a53120a955e72e6c132be5db6b44
Summary:
BSD and GNU ls behave differently in case of a missing dir:
- exit code is [1] vs [2]
- message is also slightly different
We want this to be unified for both platforms.
Test Plan:
- run the test on Linux
- hope for the best on OSX
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4921369
Summary:
Implement incremental imports.
1. Find the last imported perforce changelist.
2. Set startctx and use it in all importers
3. Import filelogs from their current position (we "should" add an additional check here, but we don't)
4. Import manifests and changelists.
Manifests are a bit tricky because we must obtain the original filelog revision
*BEFORE* we imported them, but manifest imports come "after". We could read the
most recent entry from manifests, but that won't cover the case in which files
are added. So instead we know the changelist that we are currently importing,
and looking for the rev with the correct linkrev in filelogs. That's a big ugly,
but it works. We could instead return the original offset from the worker and
pass it into the manifest importer, but I feel that is not much better and
evenutally more errorp rone.
Test Plan: cd tests && rt test-p4*
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #idi
Differential Revision: https://phabricator.intern.facebook.com/D4890110
Signature: t1:4890110:1492662991:0e141e62734e1224ac8e1c11f4e8794452455b18
Summary:
More detailed debug output for manifests. For changelog we don't
use it at the moment as changelog imports uses the perforce date, causing
changelog hashes not being stable. Will fix this in the future.
Test Plan: cd tests && rt test-p4*
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #idi
Differential Revision: https://phabricator.intern.facebook.com/D4890093
Signature: t1:4890093:1492474213:2b410da985fdd5d0786ef9dde05ecbb96f157e14
