Commit Graph

2800 Commits

Author SHA1 Message Date
David Soria Parra
07292913da p4fastimport: move p4fastimport under hgext3rd
Summary:
move p4fastimport under hgext3rd as that's where it belongs.
Also has the benefit that we package it up.

Test Plan: rt test-p4*

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5056664

Signature: t1:5056664:1494631758:4892f47922d8fbbd2c1f7793f3f29ee73fa8fa42
2017-05-12 17:03:20 -07:00
David Soria Parra
5a77207938 p4fastimport: add a debugscanlfs command that scans the repo for lfs files
Summary:
Add a debugscanlfs command to p4fastimport. We want a similar command in
lfs but in this case we also want to do the reverse lookup of a
clientspec, etc.

Test Plan: $ unbuffer hg --config 'p4fastimport.useworker=False' debugscanlfs --debug --trace --client 'dsp-test' -A

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5055890

Signature: t1:5055890:1494629600:a20abf217abe51f26b179e0ebfdaf9230e3ad34d
2017-05-12 17:03:20 -07:00
Jun Wu
d3b28f4b42 lfs: remove StoreID
Summary:
For long I have been wondering - we have `oid` already and an `oid` could
identify an object already, why do we need a `StoreID` which contains both
`oid` and `size`? It does not seem necessary?

I figured out the answer - the batch download or upload API *requires* size
information per object [1].

Since we will have LFS pointers during upload or download, we can just use
pointers. This diff removes `StoreID`, changes remote blob store to take
`pointers` directly, and local blob store to take `oid` directly.

As a side effect, `debuglfsupload` no longer takes `-o`, which seems fine
because `-r` should be more useful.

[1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md

Test Plan: `arc unit`

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy

Differential Revision: https://phabricator.intern.facebook.com/D5043694

Signature: t1:5043694:1494496522:ea10f3dd12fc2ae3ab77b8f623eeead84dcc25fa
2017-05-11 17:57:42 -07:00
Jun Wu
8641f3cc37 lfs: rename GithubPointer to gitlfspointer
Summary:
It's Git-LFS, not GitHub.
Besides, mercurial uses lowercase for class names.

Test Plan: Run existing tests

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5043570

Signature: t1:5043570:1494493515:1df24d12d8b9be51480909ef75cad56cd9e22a2e
2017-05-11 17:54:33 -07:00
Jun Wu
a024f96812 lfs: add validation to pointer
Summary:
Read the Git-LFS specification [1] and implement most checks to prevent
programming error and detect data corruption.

The new code should be stronger than what was before the refactoring, since the
old code only checks keys but not values.

[1]: https://github.com/git-lfs/git-lfs/blob/master/docs/spec.md

Test Plan: Added a new test

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5046740

Signature: t1:5046740:1494543072:2f0dbfc07cd52a10572550a277d850bdf3f78d27
2017-05-11 17:51:44 -07:00
Jun Wu
143b8c67d9 lfs: rewrite pointer logic
Summary:
Since we decided to only support GitHub's Git LFS specification, there is no
need to support multiple pointer types, so the code could be simplified.

The old code special case keys like `version`, `oid`, `hashalgo`, which makes
it longer than necessary. The new code is a rewrite treating everything as a
normal dict entry so the pointer class is much shorter: 76 -> 21 lines.

Data validation is temporary lost, which will be added back (and stronger) by
the next diff. It is separated to make review easier.


Test Plan: Run existing tests

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy

Differential Revision: https://phabricator.intern.facebook.com/D5043547

Signature: t1:5043547:1494543031:ac1100939a10a79dfd749ab6ac9c3bb7fcd84dbf
2017-05-11 17:48:21 -07:00
Jun Wu
a94a1bd72a p4fastimport: extract common logic from tests
Summary:
This diff adds a shared `p4setup.sh` that de-duplicates common logic among
tests. It also uses absolute path to make sure the extension being tested is
the version being developed.

The LFS test is also workarounded temporarily waiting for upstream change.

Test Plan: Run existing tests

Reviewers: #mercurial, davidsp

Reviewed By: davidsp

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5049279

Signature: t1:5049279:1494547832:28222fd2034115faca73860d6dd2f19206701aaa
2017-05-11 17:13:31 -07:00
Jun Wu
0cd0a0100c p4fastimport: send rlog stderr to null
Summary:
`rlog` is optional. This diff removes the following output from test output:

```
+  sh: rlog: command not found
```

Test Plan: Run existing test.

Reviewers: #mercurial, davidsp

Reviewed By: davidsp

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5047296

Signature: t1:5047296:1494535382:c57e554b57f28428a9d677d6e1089c81585e1bf2
2017-05-11 13:45:25 -07:00
Durham Goode
4a8801c1fe fixtests: update tests with new output from upstream
Upstream mutable-history has changed the output a bit. We need to update our
tests.
2017-05-11 09:13:56 -07:00
Rodrigo Damazio Bovendorp
641bb65374 githelp: making the unknown message footer configurable
Facebook probably doesn't want external users reaching out to them directly
for support, so this should be customizable to specify other possible courses
of action (especially in cases where githelp is further extended to document
git-related company-internal commands in other companies).
2017-05-11 07:19:32 -07:00
Durham Goode
fa91903ee4 treemanifest: fix flakeyness in treemanifest pushrebase test
It looks like there is some flakeyness in racing pushes during this test. Seems
to be caused by hg serve loading the initial repo state while other processes
are writing new commits, which results in a missing master. Introducing a slight
sleep to the beginning of the transaction allows all the processes to load the
correct initial state and gets rid of the flakeyness.
2017-05-10 15:30:49 -07:00
Durham Goode
8ebfa4993f treemanifest: enable stripping tree revlogs
Summary:
This hooks into the strip code to allow stripping revisions from treemanifests
as well. This will make it easier to clean up server repos when mistakes happen.

Test Plan: Adds a test

Reviewers: #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D5031489
2017-05-10 13:48:34 -07:00
Durham Goode
e079e7c52d treemanifest: fix issue with normal manifests being written as trees
Summary:
Previously, when creating a new manifestrevlog for the trees, we attempted to
copy the opener options and insert the treemanifest value for the manifestrevlog
class to pick up. It turns out that the opener we copied was actually just a
wrapper around the same underlying object that the real repo used, so when we
did opener.options = oldopener.options.copy(), we were actually putting the new
copy on both the old and new opener. This meant we enabled treemanifest for the
entire repo.

This wasn't a problem in most cases because the manifestlog had already been
loaded. But in a pushrebase world, all the data structures are refreshed when
the lock is obtained (after waiting for the other push to finish). In this
situation, the refreshed repo.manifestlog was now a treemanifest so the second
commit was pushed to the main manifest as a tree.

The fix is to not use the opener.options hack to configure the tree. Instead we
are refactoring upstream to take an explicit options to the manifestrevlog
constructor. This patch makes our internal treemanifest extension use this new
api and adds a test for push contention.

Test Plan: Added a test. It failed before, and passes after.

Reviewers: #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D5026019
2017-05-10 13:48:34 -07:00
Jun Wu
ec852f8f5f lfs: disable lfs code path if filenode is None
Summary:
`filelog.filenode` could be `None` when it's unknown - like working copy. That
breaks `isbinary` test. Let's modify `_islfs` check to return False to disable
lfs code path in that case.

Test Plan: Added a new test.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5034058

Signature: t1:5034058:1494431790:a02b69addf5d2f9b6b9a27ad71ed4f136b2bfd2b
2017-05-10 13:28:39 -07:00
Jun Wu
8b7cc1bc20 remotefilelog: fix filelog.revdiff
Summary: `revdiff` should use raw revisions. 5d11b5ed in core hg is a similar fix.

Test Plan: Added a test.

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5031916

Signature: t1:5031916:1494366439:80d496ed6f69a784d345510147eab6c479496f84
2017-05-09 14:47:58 -07:00
Durham Goode
646f9bf278 fixtests: update tests with new output
Summary:
77b66b03be changed how pushrebase and obsmarkers worked, which apparently causes
this test to change. Updating the test.

Test Plan: ran it

Reviewers: quark

Reviewed By: quark

Differential Revision: https://phabricator.intern.facebook.com/D5030830

Signature: t1:5030830:1494360593:e0c9ce9a8466cb725b3dcc22a9161c9f02b00f15
2017-05-09 13:10:11 -07:00
Jun Wu
bd12eb9379 setup: add missing mpatch library dependency to cstore
Summary:
cstore needs mpatch to function. This fixes errors like:

  # linux
  ImportError: */cstore.so: undefined symbol: mpatch_lfree
  # osx
  abort: dlopen(*/cstore.so, 2): Symbol not found: _mpatch_apply

Test Plan:
```
make clean local
cd tests
rt test-cstore.t
```

Reviewers: #sourcecontrol, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5024117

Signature: t1:5024117:1494293670:007a9f61f0175f4820d7d76de44e0399fe3ab8a9
2017-05-08 18:35:53 -07:00
Jun Wu
ee34d8ca1e lfs: remove lfs.bypass config option
Summary:
The config option was designed to be used server-side to avoid accessing remote
LFS blob stores. However, we now have `lfs.url=null:` as a clean alternative,
which will explicitly raise if the server ever tries to download remote
content. So `bypass` is no longer needed.

`bypass` could also be used for displaying raw content. That is doable using
`debugdata`.

Test Plan: Modified existing tests

Reviewers: #mercurial, rmcelroy, davidsp

Reviewed By: davidsp

Subscribers: davidsp, rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009725

Signature: t1:5009725:1494260384:332d7bd658359c004342efd01e4f13b6fc5499b1
2017-05-08 11:22:01 -07:00
Jun Wu
8909f8ded8 lfs: remove lfs.blobstore option
Summary:
`lfs.blobstore` is a path used to store local blobs. Some of the blobs are not
uploaded yet so they couldn't be easily discarded. Although blobs downloaded
from the server could be removed if necessary.

It does not make much sense to make this a config option - the data should be
stored reliably. It's also dangerous to allow write to arbitrary paths under
`repo.vfs`.

This diff makes the local blob store fixed path at `.hg/store/lfs/objects`.
The choice is similar to `.git/lfs/objects`, but with `store`, shared repo
could share their lfs stores.

Test Plan: Modified existing test.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009664

Signature: t1:5009664:1493978440:c3f6351d0ea0cca2ce3caa9f7260c5d65bcc0e5f
2017-05-08 11:21:34 -07:00
Jun Wu
186df9d185 lfs: add a command to upload lfs blobs
Summary: This makes it possible to upload lfs blobs without going through push.

Test Plan: The feature will be used in the next change.

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009622

Signature: t1:5009622:1494230385:0df33ed12c96bc91f6252398ce5091057fe32a21
2017-05-08 11:21:11 -07:00
Jun Wu
4532efb04d lfs: split prepush hook into individual functions
Summary:
This makes it possible to reuse part of them - like uploading blobs for
given revisions without going through prepush hook.

`pointer.tostoreids()` was changed to `pointer.tostoreid()` to simplify
things a bit.

Unnecessary remoterepo assignment was removed.

Test Plan: `arc unit`

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009560

Signature: t1:5009560:1494230285:6469a2701baa8cfa4511a08149a37fc429733343
2017-05-08 11:20:50 -07:00
Jun Wu
67f97aa1d3 lfs: add an integration test with lfs-test-server
Summary:
lfs-test-server is a reference implementation by GitHub [1]. Testing against it
will give us more confidence.

[1]: https://github.com/git-lfs/lfs-test-server

Test Plan: Added a test

Reviewers: #mercurial, davidsp

Reviewed By: davidsp

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009346

Signature: t1:5009346:1494042196:86e8c2e0e5c4296b1afc72efa84a9c1b9030e6fc
2017-05-08 11:13:12 -07:00
Jun Wu
3793eee754 lfs: remove 40 char length limit
Summary:
The `lfs-test-server` reference implementation [1] validates the hash, and
40-byte sha256 hexdigest will not pass that check.

[1]: https://github.com/git-lfs/lfs-test-server

Test Plan: A test will be added in the next diff.

Reviewers: #mercurial, davidsp

Reviewed By: davidsp

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009319

Signature: t1:5009319:1494006078:32950c5490935d9786553fc07d01f1fc92aacf25
2017-05-08 11:12:11 -07:00
Jun Wu
3f004c2689 lfs: ensure storeid.size is an integer
Summary:
The request JSON data used in the batch API should have `size` field as an
integer, not a string. This was discovered when testing against GitHub's
`lfs-test-server` implementation [1]. The latter was written in Golang and has
strong type requirement, it will treat `{"size": "42"}` as `{"size": 0}`.

[1]: https://github.com/git-lfs/lfs-test-server

Test Plan:
An integration test with `lfs-test-server` will be added once remaining issues
are resolved.

Reviewers: #mercurial, ikostia, davidsp

Reviewed By: ikostia, davidsp

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009248

Signature: t1:5009248:1493977299:5daf8d32cd8c933be71a41afcc7ff832eb7edb5a
2017-05-08 11:11:47 -07:00
Jun Wu
fd5d8b9996 lfs: simplify blobstore config options
Summary:
This diff simplifies lfs remote server configs to a single item: `url`,
similar to what git-lfs has.

Compare:

```
  Before                        | After
 -------------------------------+-----------------------------------
  remoteurl = http://a.com/lfs  | url = http://foo:pass@a.com/lfs
  remoteuser = foo              |
  remotepassword = pass         |
  remotestore = git-lfs         |
  ------------------------------+-----------------------------------
  remotepath = /tmp/lfs-test    | url = file:///tmp/lfs-test
  remotestore = dummy           |
```

Test Plan: Modified existing cases.

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5008882

Signature: t1:5008882:1494230125:822b0e92f45dff2a37e26e6b3e44b559b4a47e6d
2017-05-08 11:11:14 -07:00
Rodrigo Damazio
137013332e build: only add include and library directories that actually exist
On Mac, the linker complains if it's passed a -L that doesn't exist.
Checking for existance gives the most flexibility on system setup.
2017-05-08 07:53:23 -07:00
Rodrigo Damazio
136d113019 cstore: fixing implicit type conversion for building on Mac
On Mac, this currently fails with
cstore/uniondatapackstore.cpp:62:78: error: implicit conversion loses integer
      precision: 'data_offset_t' (aka 'unsigned long long') to 'size_t'
      (aka 'unsigned long') [-Werror,-Wshorten-64-to-32]
  ...ConstantStringRef((const char*)fulltextLink->delta, fulltextLink->delta_sz);
     ~~~~~~~~~~~~~~~~~                                   ~~~~~~~~~~~~~~^~~~~~~~
2017-05-08 07:49:47 -07:00
Jun Wu
2baf329dc2 tweakdefaults: let commit -M support revset
Summary: This will allow `-M` to have complex queries like `REV^'.

Test Plan: Test `scmutil.revsingle` in debugshell.

Reviewers: #mercurial, ikostia, rmcelroy

Reviewed By: ikostia, rmcelroy

Subscribers: ikostia, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5008983

Signature: t1:5008983:1493977159:7e105be95330b498d25033565be8594789406cae
2017-05-06 10:26:20 -07:00
Jun Wu
a6af2d5d27 setup: only build needed libraries
Summary:
This was reported on IRC. People building absorb got:

  lz4.h: No such file or directory

Test Plan:
`make clean`, and try `setup.py build` with:
- no parameters
- `--component absorb`, make sure lz4 is not needed
- `--component remotefilelog`, make sure it builds

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5014663

Signature: t1:5014663:1494023360:31ededc49de2dda45dfea4f62d6759cb19d819ef
2017-05-05 15:30:00 -07:00
Jun Wu
d2965bcd6a fastannotate: replace some assertions to formal errors
Summary:
fastannotate can rebuild cache automatically if it sees `CorruptedFileError` or
`CannotReuseError`, but not for `AssertionError`.

With `strip`, some assertion errors really mean file corruption. So let's use
correct exceptions so corrupted files get rebuilt automatically.

Test Plan: Run existing tests.

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5013658

Signature: t1:5013658:1494018897:a04032103b3e0cbc509adeae0d88c0a783a0dbdb
2017-05-05 14:17:35 -07:00
David Soria Parra
cd7eed4dd0 lfs: ensure svfs is set
Summary:
Some remotestores (in particular null) do not have a svfs.
Check for it before we set it.

Test Plan:
Used a nullstore with a threshold set during push and saw it
now worked.

Reviewers: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4984664
2017-05-04 16:18:32 -07:00
Jun Wu
64daf52ec8 lfs: use lfsvfs in dummy store
Summary: This simplifies code a lot.

Test Plan: Updated existing test

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5001837

Signature: t1:5001837:1493885337:eea8b5cba7234453b32eba46fd58b4d5904367bf
2017-05-04 09:25:20 -07:00
Jun Wu
fb4cd9891d lfs: remove chunking feature
Summary:
Per discussion with @davidsp, we want to stick to Git-LFS specification and
avoid non-standard behavior. The chunking behavior will happen at LFS server
transparently.

The direct motivation for this is to make it possible to implement an
efficient `filectx.cmp` that just compares hashes.

Test Plan: Updated existing test

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters, davidsp

Differential Revision: https://phabricator.intern.facebook.com/D5001827

Signature: t1:5001827:1493914639:c58694873e79a8ca910bb8ee01bf593885896664
2017-05-04 09:20:34 -07:00
Jun Wu
3d56581781 lfs: store isbinary information in LFS metadata
Summary:
Usually LFS files are binary files. But there could be exceptions. This diff
adds a new customized field `x-is-binary` to record those exceptions.

The `filectx.isbinary` API is changed to use that metadata as a fast path.

This allows us to provide a transparent user experience (whether a file is
stored in LFS or not does not affect its original `isbinary` property),
while still being able to skip loading the LFS blob if the file is binary.

Test Plan: Added a new test case

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5001117

Signature: t1:5001117:1493892406:2a8ebd42d0ae0dbae39b87b9ea16db22b99f3d12
2017-05-04 09:18:04 -07:00
Martijn Pieters
372c6c955f sparse: restore sparse profile on exception during import
Summary: Like _config, _import needs to restore the pre-existing profile if refreshing failed. Otherwise we have marked new rules as imported and applying without that actually being true.

Test Plan: rt test-sparse*

Reviewers: #sourcecontrol, rmcelroy, durham, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4985415

Tasks: 17793415

Tags: autosparse

Signature: t1:4985415:1493893462:ee4bcbf558706c3d2f58367dd7bf2533ccf1b824
2017-05-04 04:58:43 -07:00
Jun Wu
4131d4ebcd pushrebase: fix a typo in comment 2017-05-03 16:19:11 -07:00
Jun Wu
d82b8b1495 pushrebase: do not require exchange for getting markers
Summary:
D4865150 and D4934720 aren't effective in our current setup. The direct
cause in the code is because the server couldn't find common marker version:

```
  # old server-side code, returns empty in our current setup
  obsolete.commonversion(bundle2.obsmarkersversion(reply.capabilities))
```

Upon investigation, it's because there is no `exchange` enabled client-side.

But we do want one-way (server->client) markers for the rebased commits, as
long as obsstore is enabled (createmarkers is set, without exchange).

The upstream expects the server to have obsstore enabled, and exchange
enabled, to send markers. Since we are generating markers without an
obsstore (see D4865150), we are on our own way. This diff makes it one step
further.

This diff adds an explicit parameter to the `b2x:rebase` part to tell the
server what obsmarker format the client supports so the server could make a
right decision without relying on the "standard" `reply.capabilities`, which
is affected by the exchange option.

Test Plan: Change the existing test, make sure the old code fails.

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4997972

Signature: t1:4997972:1493848751:14c29654b2e8246bd12a8de8820af5b3773e2fb7
2017-05-03 16:08:35 -07:00
Jun Wu
d3f8c4e9dc lfs: override filelog.renamed code path
Summary:
An `hg pull` test triggers the following code path server-side when using
remotefilelog:

  ...
  remote:   File "/usr/lib64/python2.7/site-packages/remotefilelog/remotefilelogserver.py", line 308, in streamer
  remote:     text = _loadfileblob(repo, cachepath, path, node)
  remote:   File "/usr/lib64/python2.7/site-packages/remotefilelog/remotefilelogserver.py", line 223, in _loadfileblob
  remote:     text = createfileblob(filectx)
  remote:   File "/usr/lib64/python2.7/site-packages/remotefilelog/remotefilelogserver.py", line 348, in createfileblob
  remote:     ancestors.extend([f for f in filectx.ancestors()])
  remote:   File "/usr/lib64/python2.7/site-packages/mercurial/context.py", line 1072, in ancestors
  remote:     for parent in c.parents()[:cut]:
  remote:   File "/usr/lib64/python2.7/site-packages/mercurial/context.py", line 923, in parents
  remote:     r = fl.renamed(self._filenode)
  remote:   File "/usr/lib64/python2.7/site-packages/mercurial/filelog.py", line 62, in renamed
  remote:     t = self.revision(node)
  ...

That triggers downloading a blob. We don't want to do that server-side. So
override the `renamed` method to use LFS fast path to test rename.

Practically, this reverts part of D4906074.

Test Plan: Run existing tests. This change was made on the server.

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4992421

Signature: t1:4992421:1493802628:2bf2cf819bfed2aa61ea1c2323c03ab428732815
2017-05-03 11:10:48 -07:00
Augie Fackler
f198dbbc1e morestatus: add config knob to allow some states to be skipped
The main rationale for this change is that I've got years of use
patterns on bisect where I *start* bisect by running 'hg bisect
--reset', not *finish* with that command, so `hg status` with
morestatus enabled is spewing annoyances in nearly every repository
I've ever used. It wasn't any harder to allow people to ignore
anything they want, but (at least for now) bisect is the only state
that seems "special" in that it doesn't have a definite "end" that
Mercurial can detect and automatically clean up. I'd also be fine with
adding something like morestatus.ignorebisect if we want to avoid a
potential footgun where a user could ignore rebase state.
2017-05-03 13:34:36 -04:00
Durham Goode
0723212833 histpack: switch getnodeinfo to use common _findnode function
Summary:
It should've probably been using this function from the beginning, I just
forgot. Let's switch it to use _findnode so it takes advantage of the new
indexes.

This changes the return value of _findnode, but that's fine because the only
caller was getmissing() which didn't use the return value.

Test Plan: Ran the tests. The getnodeinfo unit test covers this code path.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4991411

Signature: t1:4991411:1493810207:7f694d41089a6efab822ae002645ed1531f6b344
2017-05-03 10:19:46 -07:00
Durham Goode
51cc08d30b histpack: update unit tests to use version 1
Summary:
The tests were still testing version 0. Let's update it to version 1 and fix the
bad version check test.

Test Plan: Ran the tests

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4991404

Signature: t1:4991404:1493802873:def4135219d731f7c0cb542012e439e79192c54f
2017-05-03 10:19:46 -07:00
Durham Goode
980bf7ecdb histpack: switch _findnode to use node index
Summary:
Now that we have node indexes, let's switch _findnode (which is responsible for
random access node lookups) to use that index.

Test Plan: Ran the tests. The getmissing unit test covers this code path.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4991400

Signature: t1:4991400:1493802781:6422be2522c3a423c80e41ce776057720b32ee98
2017-05-03 10:19:46 -07:00
Durham Goode
ef4d4cca3a histpack: move bisect to it's own function
Summary:
A future patch will be doing a similar bisect over a different index, so let's
move the bisect logic to it's own function.

Test Plan: Ran the tests

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4991393

Signature: t1:4991393:1493802679:1fb20c1e1ae6271f10953889f9d4d524d3e03a76
2017-05-03 10:19:46 -07:00
Durham Goode
bbebdda760 histpack: move entry reading to a function
Summary:
A future patch will be doing entry reading from another code path, so let's
refactor it out.

Test Plan: Ran the tests

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4991388

Signature: t1:4991388:1493802655:c20b3e394a785b31c8e94cfb0285575c6ceed70d
2017-05-03 10:19:45 -07:00
Durham Goode
b559010043 treemanifest: refactor _findnode
Summary:
Now that _getancestors is an iterator, we can get rid of the custom scan code in
_findnode.

Test Plan: Ran the tests

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4991386

Signature: t1:4991386:1493798953:e3f7faa828ef3148f8c66930d04c20c6fa83b83b
2017-05-03 10:19:45 -07:00
Durham Goode
16b8b169b7 treemanifest: return node index from _findsection
Summary:
Now that we have node indexes, let's return their location from _findsection.
Patch includes a slight rename to a constant that was poorly named as well.

Test Plan: Ran the tests

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4991385

Signature: t1:4991385:1493798906:c3147d00b6fc67b47191592032519afccf71e9af
2017-05-03 10:19:45 -07:00
Durham Goode
96ea7cd405 histpack: add per-node index
Summary:
Previously looking up a particular node in a histpack required a bisect to find
the file section, then a linear scan to find the particular node.  If you needed
to look up the latest 3000 nodes one by one, that involved 3000 linear scans,
many of which traversed the same nodes over and over.

This patch adds additional index at the end of the current histidx file. In a
future patch, we will change getnodeinfo() to use this index instead of the
linear scan logic.

Test Plan:
Ran the tests. I haven't actually verified that the data in these
indexes is correct. My next patch will add logic that reads these indexes and
will add tests around it. I won't land this until I've confirmed it's correct.

Reviewers: quark, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4983690

Signature: t1:4983690:1493798877:ae0802b4896b54bf066df9f684d94554855fd35a
2017-05-03 10:19:45 -07:00
Durham Goode
d1a927d335 packs: add entry count to pack index
Summary:
Previously, we used the length of the index file to determine the upper bounds
of the bisect. In a future patch we'll want to add more data to the end of the
index file, so we need to record how long the index portion of the index is.
This patch adds that information.

Test Plan: Ran the tests.

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4983682

Signature: t1:4983682:1493693255:57ab9af2030847fedff05b6755113ba8ce0c933b
2017-05-03 10:19:45 -07:00
Durham Goode
8674a90bb5 histpack: add version 1
Summary:
This patch just bumps the histpack version number to 1 and adds a config flag to
enable writing v1 pack files. The format hasn't actually changed in this patch,
I'm just doing the verison bump so I can update all the hashes in the tests
without working about functionality change.

In the next patch I will modify the index format, which won't affect the hashes.

Test Plan:
Ran the tests. I also ran the tests with some debug code to manually
force the sha to include 0 instead of 1 and verified that the hash didn't change
(which confirms that all of these hash changes are just because of that one byte
version change).

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4983675

Signature: t1:4983675:1493692444:5d88df4d46ce487f1b791417754ba000ecf10a1e
2017-05-03 10:19:45 -07:00
Stanislau Hlebik
8b3bb637b4 infinitepush: fix skipping downloading bundle
Summary:
`bundlefile` variable wasn't initialized if `head in nodestobundle` were true.
Even worse, the previous version of `bundlefile` were used. That means that in
some cases infinitepush sends back a wrong bundle.

Adding a unittest is quite tricky because problem shows up only in one rare
case. Example: we have a stack of 2 commits A and B and each commit in the
stack has a bookmark that points to it. Then `getbundlechunks()` `heads`
parameter contains two heads: commit A and commit B in undefined order.
If it happens that first element in the list is commit A we'll create a bundle
that contains only commit A. And we'll reuse the same bundle when we process
commit B. That means that we'll send the same bundle with commit A twice,
but no bundles with commit B.

I refactored the code to make sure we won't get the same bug in
the future.

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4969547

Tasks: 17719083, 15389402

Signature: t1:4969547:1493749313:587f9e4446a3c21b47c081a0fe4cd9e200dab5cd
2017-05-03 00:31:24 -07:00