Summary: Added -U/--unified NUM to fbshow.py, and used that to set diff.unified=NUM.
Test Plan: hg show --help; hg show -U 1; hg show -U 2; hg show -U by induction.
Reviewers: mburman, simonfar, ikostia
Differential Revision: https://phabricator.intern.facebook.com/D5052156
Tasks: 10006036
Tags: python
Summary:
The server or the HTTP library may provide more detailed error message.
Let's use them. `RequestFailedError` is also renamed to `LfsRemoteError` to
make it clear it's related to LFS and network.
Test Plan: arc unit
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5057014
Signature: t1:5057014:1494634052:00c1cb1b337a0e4cb92828cd49e95b7b0ff7e540
Summary:
move p4fastimport under hgext3rd as that's where it belongs.
Also has the benefit that we package it up.
Test Plan: rt test-p4*
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5056664
Signature: t1:5056664:1494631758:4892f47922d8fbbd2c1f7793f3f29ee73fa8fa42
Summary:
Read the Git-LFS specification [1] and implement most checks to prevent
programming error and detect data corruption.
The new code should be stronger than what was before the refactoring, since the
old code only checks keys but not values.
[1]: https://github.com/git-lfs/git-lfs/blob/master/docs/spec.md
Test Plan: Added a new test
Reviewers: #mercurial, davidsp, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5046740
Signature: t1:5046740:1494543072:2f0dbfc07cd52a10572550a277d850bdf3f78d27
Summary:
This diff adds a shared `p4setup.sh` that de-duplicates common logic among
tests. It also uses absolute path to make sure the extension being tested is
the version being developed.
The LFS test is also workarounded temporarily waiting for upstream change.
Test Plan: Run existing tests
Reviewers: #mercurial, davidsp
Reviewed By: davidsp
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5049279
Signature: t1:5049279:1494547832:28222fd2034115faca73860d6dd2f19206701aaa
Facebook probably doesn't want external users reaching out to them directly
for support, so this should be customizable to specify other possible courses
of action (especially in cases where githelp is further extended to document
git-related company-internal commands in other companies).
It looks like there is some flakeyness in racing pushes during this test. Seems
to be caused by hg serve loading the initial repo state while other processes
are writing new commits, which results in a missing master. Introducing a slight
sleep to the beginning of the transaction allows all the processes to load the
correct initial state and gets rid of the flakeyness.
Summary:
This hooks into the strip code to allow stripping revisions from treemanifests
as well. This will make it easier to clean up server repos when mistakes happen.
Test Plan: Adds a test
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D5031489
Summary:
Previously, when creating a new manifestrevlog for the trees, we attempted to
copy the opener options and insert the treemanifest value for the manifestrevlog
class to pick up. It turns out that the opener we copied was actually just a
wrapper around the same underlying object that the real repo used, so when we
did opener.options = oldopener.options.copy(), we were actually putting the new
copy on both the old and new opener. This meant we enabled treemanifest for the
entire repo.
This wasn't a problem in most cases because the manifestlog had already been
loaded. But in a pushrebase world, all the data structures are refreshed when
the lock is obtained (after waiting for the other push to finish). In this
situation, the refreshed repo.manifestlog was now a treemanifest so the second
commit was pushed to the main manifest as a tree.
The fix is to not use the opener.options hack to configure the tree. Instead we
are refactoring upstream to take an explicit options to the manifestrevlog
constructor. This patch makes our internal treemanifest extension use this new
api and adds a test for push contention.
Test Plan: Added a test. It failed before, and passes after.
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D5026019
Summary:
`filelog.filenode` could be `None` when it's unknown - like working copy. That
breaks `isbinary` test. Let's modify `_islfs` check to return False to disable
lfs code path in that case.
Test Plan: Added a new test.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5034058
Signature: t1:5034058:1494431790:a02b69addf5d2f9b6b9a27ad71ed4f136b2bfd2b
Summary: `revdiff` should use raw revisions. 5d11b5ed in core hg is a similar fix.
Test Plan: Added a test.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5031916
Signature: t1:5031916:1494366439:80d496ed6f69a784d345510147eab6c479496f84
Summary:
77b66b03be changed how pushrebase and obsmarkers worked, which apparently causes
this test to change. Updating the test.
Test Plan: ran it
Reviewers: quark
Reviewed By: quark
Differential Revision: https://phabricator.intern.facebook.com/D5030830
Signature: t1:5030830:1494360593:e0c9ce9a8466cb725b3dcc22a9161c9f02b00f15
Summary:
The config option was designed to be used server-side to avoid accessing remote
LFS blob stores. However, we now have `lfs.url=null:` as a clean alternative,
which will explicitly raise if the server ever tries to download remote
content. So `bypass` is no longer needed.
`bypass` could also be used for displaying raw content. That is doable using
`debugdata`.
Test Plan: Modified existing tests
Reviewers: #mercurial, rmcelroy, davidsp
Reviewed By: davidsp
Subscribers: davidsp, rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5009725
Signature: t1:5009725:1494260384:332d7bd658359c004342efd01e4f13b6fc5499b1
Summary:
`lfs.blobstore` is a path used to store local blobs. Some of the blobs are not
uploaded yet so they couldn't be easily discarded. Although blobs downloaded
from the server could be removed if necessary.
It does not make much sense to make this a config option - the data should be
stored reliably. It's also dangerous to allow write to arbitrary paths under
`repo.vfs`.
This diff makes the local blob store fixed path at `.hg/store/lfs/objects`.
The choice is similar to `.git/lfs/objects`, but with `store`, shared repo
could share their lfs stores.
Test Plan: Modified existing test.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5009664
Signature: t1:5009664:1493978440:c3f6351d0ea0cca2ce3caa9f7260c5d65bcc0e5f
Summary:
lfs-test-server is a reference implementation by GitHub [1]. Testing against it
will give us more confidence.
[1]: https://github.com/git-lfs/lfs-test-server
Test Plan: Added a test
Reviewers: #mercurial, davidsp
Reviewed By: davidsp
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5009346
Signature: t1:5009346:1494042196:86e8c2e0e5c4296b1afc72efa84a9c1b9030e6fc
Summary:
Per discussion with @davidsp, we want to stick to Git-LFS specification and
avoid non-standard behavior. The chunking behavior will happen at LFS server
transparently.
The direct motivation for this is to make it possible to implement an
efficient `filectx.cmp` that just compares hashes.
Test Plan: Updated existing test
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters, davidsp
Differential Revision: https://phabricator.intern.facebook.com/D5001827
Signature: t1:5001827:1493914639:c58694873e79a8ca910bb8ee01bf593885896664
Summary:
Usually LFS files are binary files. But there could be exceptions. This diff
adds a new customized field `x-is-binary` to record those exceptions.
The `filectx.isbinary` API is changed to use that metadata as a fast path.
This allows us to provide a transparent user experience (whether a file is
stored in LFS or not does not affect its original `isbinary` property),
while still being able to skip loading the LFS blob if the file is binary.
Test Plan: Added a new test case
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5001117
Signature: t1:5001117:1493892406:2a8ebd42d0ae0dbae39b87b9ea16db22b99f3d12
Summary: Like _config, _import needs to restore the pre-existing profile if refreshing failed. Otherwise we have marked new rules as imported and applying without that actually being true.
Test Plan: rt test-sparse*
Reviewers: #sourcecontrol, rmcelroy, durham, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4985415
Tasks: 17793415
Tags: autosparse
Signature: t1:4985415:1493893462:ee4bcbf558706c3d2f58367dd7bf2533ccf1b824
Summary:
D4865150 and D4934720 aren't effective in our current setup. The direct
cause in the code is because the server couldn't find common marker version:
```
# old server-side code, returns empty in our current setup
obsolete.commonversion(bundle2.obsmarkersversion(reply.capabilities))
```
Upon investigation, it's because there is no `exchange` enabled client-side.
But we do want one-way (server->client) markers for the rebased commits, as
long as obsstore is enabled (createmarkers is set, without exchange).
The upstream expects the server to have obsstore enabled, and exchange
enabled, to send markers. Since we are generating markers without an
obsstore (see D4865150), we are on our own way. This diff makes it one step
further.
This diff adds an explicit parameter to the `b2x:rebase` part to tell the
server what obsmarker format the client supports so the server could make a
right decision without relying on the "standard" `reply.capabilities`, which
is affected by the exchange option.
Test Plan: Change the existing test, make sure the old code fails.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4997972
Signature: t1:4997972:1493848751:14c29654b2e8246bd12a8de8820af5b3773e2fb7
The main rationale for this change is that I've got years of use
patterns on bisect where I *start* bisect by running 'hg bisect
--reset', not *finish* with that command, so `hg status` with
morestatus enabled is spewing annoyances in nearly every repository
I've ever used. It wasn't any harder to allow people to ignore
anything they want, but (at least for now) bisect is the only state
that seems "special" in that it doesn't have a definite "end" that
Mercurial can detect and automatically clean up. I'd also be fine with
adding something like morestatus.ignorebisect if we want to avoid a
potential footgun where a user could ignore rebase state.
Summary:
The tests were still testing version 0. Let's update it to version 1 and fix the
bad version check test.
Test Plan: Ran the tests
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4991404
Signature: t1:4991404:1493802873:def4135219d731f7c0cb542012e439e79192c54f
Summary:
Previously looking up a particular node in a histpack required a bisect to find
the file section, then a linear scan to find the particular node. If you needed
to look up the latest 3000 nodes one by one, that involved 3000 linear scans,
many of which traversed the same nodes over and over.
This patch adds additional index at the end of the current histidx file. In a
future patch, we will change getnodeinfo() to use this index instead of the
linear scan logic.
Test Plan:
Ran the tests. I haven't actually verified that the data in these
indexes is correct. My next patch will add logic that reads these indexes and
will add tests around it. I won't land this until I've confirmed it's correct.
Reviewers: quark, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4983690
Signature: t1:4983690:1493798877:ae0802b4896b54bf066df9f684d94554855fd35a
Summary:
Previously, we used the length of the index file to determine the upper bounds
of the bisect. In a future patch we'll want to add more data to the end of the
index file, so we need to record how long the index portion of the index is.
This patch adds that information.
Test Plan: Ran the tests.
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4983682
Signature: t1:4983682:1493693255:57ab9af2030847fedff05b6755113ba8ce0c933b
Summary:
This patch just bumps the histpack version number to 1 and adds a config flag to
enable writing v1 pack files. The format hasn't actually changed in this patch,
I'm just doing the verison bump so I can update all the hashes in the tests
without working about functionality change.
In the next patch I will modify the index format, which won't affect the hashes.
Test Plan:
Ran the tests. I also ran the tests with some debug code to manually
force the sha to include 0 instead of 1 and verified that the hash didn't change
(which confirms that all of these hash changes are just because of that one byte
version change).
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4983675
Signature: t1:4983675:1493692444:5d88df4d46ce487f1b791417754ba000ecf10a1e
Summary:
The newly changed `LZ4_decompress_safe` is unhappy about empty buffer. So let's
check empty revision explicitly.
Test Plan: Added a test
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4986277
Signature: t1:4986277:1493743735:be2ad6047bb0b983425c6e49b7c5ddf2c94d1c1a
Summary:
This test won't be run unless it is in the right directory.
#m brownpaperbag
Test Plan: run arc unit
Reviewers: #sourcecontrol, jeroenv
Reviewed By: jeroenv
Subscribers: jeroenv, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4985738
Tasks: 17796162
Signature: t1:4985738:1493736355:4c9358fbf536661dc6b1f11c546332cda54f21b2
Summary:
Support the action archive. 'archive' means that a revision was
"archived" to a different depot. We must ensure we support the action correctly
in order to have a smooth import.
Test Plan: run it && rt test-p4* test-check*
Reviewers: #sourcecontrol, #idi, wlis
Reviewed By: wlis
Subscribers: wlis, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4980571
Signature: t1:4980571:1493676115:ad0d3748c52747aeb6427fd25ece9d4987886936
Summary:
When passing --limit we are processing N Perforce changelists at a time.
The goal is to provide savepoints for large imports.
Test Plan: rt test-p4* test-check*
Reviewers: #sourcecontrol, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4980482
Signature: t1:4980482:1493688071:800a0bafda33a17cb2ef54c9f399db7055a8cbf9
Summary:
This diff makes 2 changes to v1 packfile metadata:
1. Move `key` in a metadata entry to before `size`.
```
old: [entry-size: 2 byte] [key: 1 byte] [data: var length]
new: [key: 1 byte] [data-size: 2 byte] [data: var length]
```
Previously `entry-size == 0` does not make sense.
2. Use binary to represent sizes, instead of ASCII.
Related utility methods are cleaned up a bit so it's harder to make mistakes.
Test Plan: Updated existing tests
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4983189
Signature: t1:4983189:1493689852:22d544d73ed63fac83f849786de035af304161ce
Summary:
This shows how much overhead the `getmeta` call introduces.
Initial investigation shows the C version is about 10x as fast as than `get` or `getdeltachain`.
Test Plan: Run `hg perfdatapack` in an repo.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4909157
Signature: t1:4909157:1492667211:e82a14e12d04f60c0cee4124e83845ec166a78cc
Summary:
This diff implements getmeta in C and enables related tests.
Now all content stores support `getmeta`.
Test Plan: Run existing tests.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4960926
Signature: t1:4960926:1493611048:55095c32927fac74e698f21d47173cb8a7523fb6
The bundle2 part was always being generated, even for non-treemanifest repos
(because fastmanifest imports the treemanifest module, so the b2partsgenerator
gets registered). This cause pushes to break in any repo with fastmanifest
enabled.
Summary: We've made changes to arcanist configuration recently. Need to do the same in the tests.
Test Plan: arc unit
Reviewers: #scmquery, rmcelroy, ikostia
Reviewed By: ikostia
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4969670
Signature: t1:4969670:1493376937:536470162c52c8be953c813c1fea54f3e506f1e9
Summary:
To enable pushing between peers (and eventually pushing to the server), let's
teach bundle creation to include the trees being pushed.
Test Plan: Adds a test
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4957456
Signature: t1:4957456:1493266296:67f98a2b3d691644bde9098a713d05266f349cde
Summary:
Previously, history stores only had getancestors() apis which returned all the
ancestors. This was expensive if there was a lot of ancestors, like for the root
tree of treemanifests. Let's add an api for accessing a single history entry.
This will be useful in pack generation for only fetching the history related to
the trees we're sending at that time.
Test Plan: Added a test
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4957432
Signature: t1:4957432:1493263124:a155ac5a70c35f7e25a5cc48c9d9c2126d4c5858
Summary:
Previously, a local commit would only write data packs, and it just threw away
the history data entirely. Let's add history packs and record them on commit.
Test Plan:
The tests are updated to show these new packs. In some cases the
datapacks got smaller as well, since they can now take advantage of history data
for delta choices.
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4956105
Signature: t1:4956105:1493265399:d3fa1052c207fba0045cbb92b4d833d18d48e099
Summary:
Previously, the logic that added data to a mutable history pack was required to
add it in the correct order (all entries for a certain file at once, and in
newest-first order). This required the callers to jump through weird hoops if
the data came in out of order or at different times in the transaction.
This patch moves the ordering logic to be inside MutableHistoryPack, so callers
can add the data in any order they wish, and it will get sorted before being
serialized.
This does add memory pressure to things that read a lot of history, like repack.
If this becomes a problem we may want to add a 'historypack.flush()' api that
let's us tell the history pack it's ok to flush it's current contents to disk.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4956096
Signature: t1:4956096:1493264693:a2275a49e35565d4b11244e3e5dd82c25de7e16e
Summary:
Adds a config option that will automatically download the last N trees from the
server during pull. It efficiently decides what bases to set, based on what
commit parents it already has.
Test Plan: Added a test
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4923211
Signature: t1:4923211:1493200730:39872db6aabb48c12131ff8501a950c8ee60e2b0
Summary:
This allows a third-party tool like Buck report more detail on what a sparse
command actually achieved.
Test Plan: Run the test suite.
Reviewers: #sourcecontrol, durham, rmcelroy, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4962550
Tasks: 17150508
Tags: autosparse
Signature: t1:4962550:1493307403:d14e16d079121688a4eb69f0d69cd02ab55c4936
Summary:
The test covers common workflows like clone, commit, push, update, pull. It
exercises the remotefilelog plain store and Python datapack store to make sure
they won't lose the revlog flag. The test also tries to verify rename works
correctly.
Since the lfs extension may be eventually upstreamed, it seems a good idea to
make remotefilelog call `lfs.wrapfilelog` so lfs is free from remotefilelog
code.
Test Plan: Added a test
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4904281
Signature: t1:4904281:1492560308:5fd9f214ada6de795735ea7d737d30c1bf39812a
Summary:
This diffs add a `getmeta` method to all content stores. The cdatapack code is
modified to pass the tests, it needs further change to support `getmeta`.
The datapack format is bumped to v1 from v0. For v1, we append a `metadata`
dict at the end of each revision. The dict is currently used to store revlog
flags and rawsize of raw revlog fulltext. In the future we can put more data
like a second hash etc, without changing API or format again.
This diff focuses on correctness. A datapack caching layer to speed up
`getmeta` will be added later.
Tests are updated since we write new v1 packfile now and the format change
leads to different content and packfile names.
`Makefile`, `ls-l.py` are added to make tests easier to maintain.
Test Plan: Updated existing tests.
Reviewers: #mercurial, rmcelroy, durham
Reviewed By: durham
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4903917
Signature: t1:4903917:1493255844:7ef5d487096cd2f78f2aaae672a68d49f33632ee
Summary: Today is a bad day. I don't even know how this got in there.
Test Plan: y
Reviewers: durham, #mercurial
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4957784
Summary: Otherwise, the test requires hg-dev and will break third-party developers.
Test Plan: Reran test
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4957662
Signature: t1:4957662:1493242115:7e83b130791b07b1e45f04ce199e5c4ee9d58d77
Summary:
allows users to have JSON progress bar information written to a path
Controlled by the `ui.progressfile` config. Mercurial will overwrite this file
each time the progress bar is updated.
The schema of this file is (JSON):
- topics: array of topics from oldest to newest. (last is always the active one)
- state: map of topic names to objects with keys:
- topic (e.g. "changesets", "manifests")
- pos: which item number out of <total> we're processing
- total: total number of items
- unit: name of the type of unit being processed (e.g., "changeset")
- item: the active item being processed (e.g., "changeset #5")
- active: whether this is the currently active progress bar
- units_per_sec: if active, how many <unit>s per sec we're processing
- speed_str: if active, a human-readable string of how many <unit>s per sec
we're processing
- estimate_sec: an estimate of how much time is left, in seconds
- estimate_str: if active, a human-readable string estimate of how much time
is left (e.g. "2m30s")
Test Plan: added a test
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: rmcelroy, quark, stash, asriram, mjpieters, durham
Differential Revision: https://phabricator.intern.facebook.com/D4752788
Signature: t1:4752788:1493233878:b49095237d32233c78cd0e0aaaa7b94e0e9e7011
