Summary:
Show the 'impact' of a profile, relative to a non-sparse working copy.
By default, this is the percentage of the total file count; adding --verbose will show the file sizes too.
The defined matchers have been refactored to reuse more of mercurial.match.basematcher, making it easier to reuse these in a wider mercurial context and avoiding repetition of common methods.
Reviewed By: quark-zju
Differential Revision: D7415720
fbshipit-source-id: 4ac3492c61aa70ee71d4bdf8c201b905a345a9d1
Summary:
Previously we relied on the list of files created by strip to decide
what directories to strip. It turns out this list wasn't adequate though, since
it's possible for directories to change even when files do not. For example,
during a merge if different files were added on different sides of the merge,
the files don't change during the merge but their containing directory does.
This diff special cases merge commits to make sure all the directories affected
by the merge are included in the strip.
Reviewed By: quark-zju
Differential Revision: D7409156
fbshipit-source-id: 9bf67eefb70189300c29db60d9945a7f608dfdda
Summary:
Previously we were trying to send trees to all clients during an
infinitepull, even ones that didn't support treemanifest. This caused
infinitepulls that required rebundling to fail for non-tree clients.
The fix is to just not send them unless the client is advertising the
capability.
Reviewed By: phillco
Differential Revision: D7432374
fbshipit-source-id: 1fae14a158ef56fe39439a718b1b98928f4e07b0
Summary: The order of the two reported files can differ, depending on the random hash seed.
Reviewed By: ryanmce
Differential Revision: D7427725
fbshipit-source-id: 9ec76867877553a02d1a19642d98d626b0ba91cc
Summary:
I need to be able to test for the presence of a cache without the ability to use a function, and I rather not use a sentinel value or have func raise an exception.
Note that the extension already is vulnerable to race conditions (there is plenty of time between the get, the func() call, and the set as it is).
Also, make it trivial to use JSON as a serializer.
Reviewed By: quark-zju
Differential Revision: D7415721
fbshipit-source-id: dc06e8f3efe725858a7960f7acfecc5a60390b85
Summary:
This gives us access to the specifics of a subcommand, including switches
(still stubbed out). Note: `hg sparse <subcommand> -h` doesn't work yet, as the
`-h` command line switch would need intercepting as well.
Reviewed By: quark-zju
Differential Revision: D7413859
fbshipit-source-id: 02534f1255a11985ebe8636b26946fbeffc6fe09
Summary:
Add the ability to set a `formatfunc` on a progres bar, which formats the
numbers used. Use `util.bytecount` as a format function in the places
where we have progress bars in numbers of bytes.
Reviewed By: quark-zju
Differential Revision: D7355232
fbshipit-source-id: 117c7035d46d47259cdfd70b80438cc6f4615977
Summary:
When updating a working copy that is limited by a sparse profile, do not abort on case collisions outside of the current working copy view.
Performance-wise, this isn't a big change; previously `_checkcollision()` would convert the whole manifest to a set, now we filter first with the matcher, then convert the remainder to a set. Either the original set conversion or the filtering can take O(size-of-manifest) time.
Reviewed By: ryanmce
Differential Revision: D7350251
fbshipit-source-id: efb8e270631f0aaa75c34c7b68189c60efe45984
Summary:
D7397947 broke `test-check-code.t` which complains about the use of
alias instead of function. This is blocking our continuous builds. This commit
addresses the same.
Reviewed By: quark-zju
Differential Revision: D7404361
fbshipit-source-id: 7907e913327c77861258a37ff8b87d467a7a15c6
Summary: This script can be used to generate a large number of commits.
Reviewed By: DurhamG
Differential Revision: D7363243
fbshipit-source-id: 1a1e3ba7fca29dc695446eeffb715474ea2943ad
Summary: This shows basic details about a template, including base metadata. Once we have support for per-subcommand switches we can add further filtering.
Reviewed By: ryanmce
Differential Revision: D7365794
fbshipit-source-id: 3dd362f099e7f4d6db73c9bda7c24a9f4f2f90ee
Summary: Now all commands can be subcommands. The command-line switches will be hidden in a next commit (but remain available for backwards compatibility).
Reviewed By: ryanmce
Differential Revision: D7365393
fbshipit-source-id: 29b8ed2ac6bffccda41ed3f738087dde23312a8c
Summary:
This means we can
- count on the minirst module to format longer lines correctly,
- show subcommands when an error occurs (so full is false),
- add verbose help when the -v switch
Reviewed By: ryanmce
Differential Revision: D7365392
fbshipit-source-id: c8711bd19a6d992ad86f24f3c07b381f69d02121
Summary:
obsshelve: highlight shelved changes in smartlog (you can see with --hidden)
we highlighted only active shelved changes, those that has been unshelved are just ordinary hidden changes.
Reviewed By: ryanmce
Differential Revision: D7383800
fbshipit-source-id: 2df4092514d58315a6a204411feed99819df5c93
Summary:
Allow color config to be something like `color132:red` that is to use 256
color if supported, but fallback to `red`.
To get a consistent test output. `run-tests.py` is changed to disable 256
color support explicitly.
Reviewed By: singhsrb
Differential Revision: D7388277
fbshipit-source-id: da74ae8fc70c971901d56a6985976db44cbec0d9
Summary:
As suggested by wez, "dim" is less supported than the bright colors.
So let's use the bright versions instead. Basically, replacing "red",
"red dim" with "brightred" and "red".
Reviewed By: singhsrb
Differential Revision: D7387254
fbshipit-source-id: f741d9e1c12115acdd1a3a2cae1658d8fae534bf
Summary:
The reason why we would like to have a real user there is that
we may want to start push shelved changes in commit cloud, so
we should have correct user on the commit
Reviewed By: ryanmce
Differential Revision: D7380732
fbshipit-source-id: 27b07f992a3e394016ce69e1dc9694a4dd1c336b
Summary:
This is based on the hg show implementation. hg sparse --list-profiles is not yet widely known, so now is the time to move it to a subcommand.
This is the first step in untangling the mess that is the `hg sparse` forest of options.
Currently, all switches on the command are mutually exclusive, except for `—force` and `—template`, which each only apply to a subset of the actions the other switches affect.
Subcommands are the right pattern for mutually-exclusive actions that can accept their own individual switches.
Reviewed By: quark-zju
Differential Revision: D7350928
fbshipit-source-id: d03014cf7edd2f089f670d11465c70940d96c070
Summary:
Previously we were storing the changelog on the manifestlog and using
it to resolve linkrevs before serializing them. It turns out the changelog can
be invalidated at a different rate than the manifestlog, so we could encounter
issues where the manifestlog held a reference to the old changelog.
To fix this, let's hold a reference to the repo and access the changelog from
there when we need it. This introduces a circular reference between the
manifestlog and the repo, but it's probably fine for now until we can get rid of
the need for changelog invalidation.
Reviewed By: singhsrb
Differential Revision: D7360321
fbshipit-source-id: 2317c7fcd6b307a50b64f0c5df97dda2955f3e21
Summary:
Remove ui.progress as a method of updating progress. All progress bars now go
through new-style progress bars.
This also splits out the rendering of progress bars from the reporting of
progress. All tests are updated to use new-style debug progress bars, which
simply report the position of the progress bar. Rendering of progress bars
will be tested separately once the progress bar engine has been rewritten.
Reviewed By: quark-zju
Differential Revision: D7329488
fbshipit-source-id: 14f8ab67365ddd98b74986aa25d9abc7a0546144
Summary:
Update hg.copystore and util.copyfiles to use the new-style progress bar
context manager.
Reviewed By: ryanmce
Differential Revision: D7329482
fbshipit-source-id: 4ac1def57e0ae4e7819c7c0c4d6f1715b8cbb625
Summary:
Previously, when looking for a treemanifest we'd inspect the flat
manifest revlog to get a hint for which nearby manifests might exist. This
assumed the data was in the flat manifest though, and if it wasn't it threw an
exception which caused us to not look at the server for the tree.
This commit fixes the case where the flat manifest doesn't have the commit, so
it can still look at the server.
Reviewed By: quark-zju
Differential Revision: D7343608
fbshipit-source-id: 57d5da9a8b8d71e33df9a8a0647b7bfe8815e37b
Summary:
Tests pushrebase can preserve commit hash if rewritedates is set to False.
This is the desired property mentioned in T26625986.
Reviewed By: DurhamG
Differential Revision: D7280498
fbshipit-source-id: e8943ddfec55a524c5cfd1dc9de765eace1e5296
Summary:
We have some repo's whose first commit didn't not add any files, which
means they have nullid as their manifest pointer which breaks backfilltrees.
This patch fixes that.
Reviewed By: singhsrb
Differential Revision: D7340678
fbshipit-source-id: 85b048b81c1861fd48d5cc082d414271aee7283b
Summary: Although the old glob *should* work, in practice the treemanifest matcher seems to have issues with glob:**/ paths, where relglob: works.
Reviewed By: ryanmce
Differential Revision: D7287992
fbshipit-source-id: d3802e730d937eeca7059dac691d8dac5a5de262
Summary:
This adds the ability to specify a config file to be used during the
command. This is useful during clones for letting the clone command use the
given repositories system specified repo-specific hgrc file.
Reviewed By: quark-zju
Differential Revision: D7311576
fbshipit-source-id: a97d8ebada2e0bea27c75a7650df8ede00dc10c6
Summary:
worddiff needs extra computation and does not have effect if color is
disabled. It should also be disabled with HGPLAIN=1.
Reviewed By: ryanmce
Differential Revision: D7315276
fbshipit-source-id: b59255bc708713186bd4ddafdc469763e1a66ea6
Summary:
There were recent complains about both quality [1] [2] and performance [3]
of the current word diff algorithm.
The current algorithm is actually bad in various ways:
- Lines could be matched across hunks, which is confusing (report [1]).
- For short lines, they can fail "similarity" check, which means they
won't be highlighted when they are expected to be (report [2]).
- Various performance issues:
- Using difflib implemented by pure Python, which is both slow and
suboptimal comparing with xdiff.
- Searching for matched lines across hunks could be O(N^2) if there are
no match found.
Thinking it in a "highlight" way is actually tricky, consider the following
change:
```
# before
foo = 10
# after
if True:
foo = 21 + 3
```
It's obvious that "10" and "21 + 3" need highlighting because they are
different. But what about "if True:"? In theory it's also "different" and
need highlighting. How about purely inserted or deleted hunks then?
Highlighting all of them would be too noisy.
This diff rewrites the word diff algorithm. It differs in multiple ways:
1. Get rid of "matching lines by similarity" step.
2. Only diff words within a same hunk.
3. Dim unchanged words. Instead of highlighting changed words.
4. Treat pure insertion or deletion hunks differently - do not dim or
highlight words in them.
5. Use xdiff instead.
6. Use a better regexp to split words. This reduces the number of tokens sent
to the diff algorithm.
1, 2, 5, 6 help performance. 1, 2, 3, 4 make the result more predictable and
trustworthy. 3 avoids the nasty question about what to highlight. 3 and 4 makes
it more flexible for people to tweak colors. 6 makes the result better since it
merges multiple space tokens into one so xdiff will less likely miss important
matches (than meaningless matches like spaces).
"bold" and "underline" were removed so the changed words will have regular
red/green colors. The output won't be too "noisy" even in cases where code are
changed in a way that inline word matching is meaningless. For people who want
more contrast, they can set:
[color]
diff.inserted.changed = green bold
diff.deleted.changed = red bold
Practically, when diffing D7319718, the old code spends 4 seconds on finding
matched lines preparing for worddiff:
```
| diffordiffstat cmdutil.py:1522
\ difflabel (17467 times) patch.py:2471
....
> 3927 \ _findmatches (22 times) patch.py:2537
348 \ __init__ (8158 times) difflib.py:154
340 | set_seqs (8158 times) difflib.py:223
328 | set_seq2 (8158 times) difflib.py:261
322 | __chain_b (8158 times) difflib.py:306
1818 \ ratio (8158 times) difflib.py:636
1777 | get_matching_blocks (8158 times) difflib.py:460
1605 \ find_longest_match (51966 times) difflib.py:350
38 | __new__ (51966 times) <string>:8
29 \ _make (36035 times) <string>:12
143 \ write (17466 times) ui.py:883
```
The new code takes 0.14 seconds:
```
| diffordiffstat cmdutil.py:1522
\ difflabel (23401 times) patch.py:2562
....
> 140 \ consumehunkbuffer (23346 times) patch.py:2585
130 | diffsinglehunkinline (23240 times) patch.py:2496
215 \ write (23400 times) ui.py:883
118 \ flush cmdutil.py:1606
118 | write ui.py:883
```
[1]: https://fburl.com/lkb9rc9m
[2]: https://fburl.com/0r9bqf0e
[3]: https://fburl.com/pxqznw31
Reviewed By: ryanmce
Differential Revision: D7314726
fbshipit-source-id: becd979cb9ac3fd3f4adae11cb10804d535f58df
Summary:
Instead of yielding tokens directly, buffer them if they belong to a same
hunk. This makes it easier for the upcoming new worddiff algorithm to only
focus on the diff hunk, instead of having to worry about other contents.
This breaks how the existing experimental worddiff algorithm works, so the
algorithm was removed, and related tests are disabled for now.
Reviewed By: ryanmce
Differential Revision: D7314725
fbshipit-source-id: 344e502cd185b2412fbd2ee299131bbb4560ac17
Summary:
A prior diff optimized push/pull to not send public trees between
peers, since those trees can be downloaded from the main server. Let's be
careful when sending data to the main server and always send everything.
In the future we should add validation on the server that the received data is
complete, but Mercurial doesn't currently do that today.
Differential Revision: D7296253
fbshipit-source-id: 49513685d19991a70d66da1d734ddae23491ed0c
Summary:
This allows pushing a treeonly pack to a server without using
pushrebase.
Differential Revision: D7295686
fbshipit-source-id: b0bfe4fbb04bc765e57f1db82909fa1ae7b3063b
Summary:
It can sometimes be nice to be able to just repack loose data and avoid existing packs.
This came up during building the megarepo, which creates many loose files, but including the packs added too much repack time.
Differential Revision: D7209296
fbshipit-source-id: 10afed40e409733e0ee004f025013cf86f3f7bf6
Summary: Help people debug issues by providing useful hints when templates fail to parse.
Differential Revision: D7148780
fbshipit-source-id: bd2b0f2bdeb0c970732fcaf53854d4a003b08571
Summary:
This logic is largely based on the similar logic added to template error
messages in D2608 and D2609, but with a few tweaks based on how revsets
actually work.
Differential Revision: D7148779
fbshipit-source-id: fb435788711a2c4ef881dfaeac5176fc1972c07a
Summary:
Previously we were just putting nullid as the linknode in client side
trees, because when the trees were added the changelog hadn't been written yet,
so we didn't know the linknode. This diff updates manifestlog.add to pass
linkrevs to mutablehistorypack, which get resolved at serialization time into
the appropriate linknode.
Reviewed By: ryanmce
Differential Revision: D7280104
fbshipit-source-id: bbc8a7bfad31da296a1b65973875ba2e1e1f7a95
Summary:
Previously we were just putting nullid as the linknode in client side
trees, because when the trees were added the changelog hadn't been written yet,
so we didn't know the linknode. This diff updates mutablehistorypack to allow
resolving the linknodes at serialization time instead of add time. This will be
used in a future diff to fix storing linknodes in trees.
Reviewed By: ryanmce
Differential Revision: D7280105
fbshipit-source-id: 70063e627d0fd7baeb017bac5ac55957a100d06c
Summary:
Previously converttotrees relied on being passed a linkrev to pass to
manifestlog.add(). If a linkrev wasn't provided though, it would pass None and
no linknode was stored. If a linknode is available, let's prefer that, and only
use the linkrev when we have to.
Reviewed By: ryanmce
Differential Revision: D7280109
fbshipit-source-id: d17015789a213fab6ff57563216e6943d2525fee
Summary:
Previously pushrebase pushes would send a tree pack part as well as a
rebase pack part. The tree pack part was ignored. Let's make receiving tree pack
parts on the server an error, and instead make the client not send them if it's
doing a pushrebase push.
A future diff will come back and add support for non-pushrebase push of tree
packs. This diff just makes use sure that the new logic won't run during
pushrebase.
Reviewed By: ryanmce
Differential Revision: D7267802
fbshipit-source-id: cabff989c178afb3706b77a03a299bcf3195f289
Summary:
Currently infinitepush relies on the ability to send flat manifests to
tree clients and the clients doing the conversion. This relies on a hack where
we don't pass the bundlecaps to the infinitepush changegroup creator, so it
doesn't realize it's talking to a treeonly client and therefore sends flat
manifests to a treeonly client. This is hacky. To fix this, let's start by
making it possible for the server to generate trees on demand from the flat
manifests. This will also be useful later when the server is treeonly, so it can
ondemand generate old flat bundles into a treeonly one.
A later diff will then pass the appropriate bundlecaps to the rebundle
changegroup function.
Reviewed By: ryanmce
Differential Revision: D7256554
fbshipit-source-id: 40d5d9537a5e11b3d186c8ec9a2579261f1b772e
Summary:
Add a new extension, `clienttelemetry`, which sends information about the
client to the server. This information is logged by the server, which can use
the `sampling` extension to store this information for later analysis.
Includes a few basic pieces of information by default, including a correlator
which can be used to correlate logs from the client and the server.
Reviewed By: mjpieters
Differential Revision: D7271508
fbshipit-source-id: f152fe177d8bc73179d3a2c4d0d070d468885e2a
Summary:
I somehow had a brain fart before landing my infinitepush phases diff
and replaced a hash with a rev number, even though rev numbers don't work with
infinitepush. This fixes that.
fbshipit-source-id: aa85614d40e173f6472cb9570af8510eb15ea353
Summary:
Add obsmarkers to the things that are synced.
For the LocalService, obsmarkers are stored base64-encoded using version 1 of
the obsmarker binary format. This is sufficient for unit tests.
Differential Revision: D7181399
fbshipit-source-id: 61377105986de561622a160134d20fdbd54bb88f
Summary:
Adds the initial skeleton of the `commitcloud` extension to Mercurial, which
will synchronise heads and bookmarks (and later, obsmarkers) with commit cloud.
Current limitations:
- only supports a stub version of the service that uses local files
- requires manual pushbackup to send commits to the server via infinitepush
- doesn't do anything with obsmarkers
Differential Revision: D7167715
fbshipit-source-id: 53443f3fb8dff0fe52257d7b20fc601dbe10c883
Summary: Put titles, if present, into a second 'column' in the plain output.
Differential Revision: D7258046
fbshipit-source-id: dedbb04ecbf0b2262c4c507683262e4b7bef8d70
Summary:
Profiles can now have a [metadata] section, with key-value pairs defining some metadata to be associated with the profile.
This supports multiline entries by starting a continuation line with some whitespace:
```
[metadata]
title = Some cool title
description = A longer description
will be joined with newlines
provided you start with a space
or a tab.
```
Differential Revision: D7257628
fbshipit-source-id: d6b09161d54cb6824f691d07caee83d0f8bd0a84
Summary: We regularly parse multiple profiles, through includes or when discovering. Be clear what profile file caused the issue.
Differential Revision: D7257636
fbshipit-source-id: 1203c13f6c6a4e4d1397ace818aee21b9db38550
Summary:
infinitepush manipulates what phases are sent back to the client. The
old behavior was that it would report every commit from the bundle to the client
as draft, even if the commit wasn't related to the heads being fetched and even
if the commit wasn't actually a root. The mercurial client logic expected these
values to actually be roots, and this caused some logic to act incorrectly and
mark commits unrelated to the pulled head public.
The fix is to only send roots, and only ancestors of the requested head.
Reviewed By: singhsrb
Differential Revision: D7265155
fbshipit-source-id: 709c3c807258049e44e493b0668f3034a0b11328
Summary:
There's a bug in infinitepush where it can cause client phases to get
screwed up. This affects pulls where multiple stacks were uploaded at once, then
only part of the stack is pulled. Let's add a test to expose the bug.
Reviewed By: singhsrb
Differential Revision: D7265104
fbshipit-source-id: ec1320d46ec865e8fbcf324e8d94737a5c111ef7
Summary: Delete an unnecessary test command which delete a non-existed local bookmark.
Reviewed By: StanislavGlebik
Differential Revision: D7216466
fbshipit-source-id: f8fc4215ef1fa725c5e5f92ef03cc2616326ac59
Summary:
Previously, if we weren't sending trees we would not attempt to process
what files needed to be sent. This was incorrect, since files may be sent
independent of what trees are decided to be sent. So let's update the code to
care about shouldaddfilegroup instead of cansendtrees.
This exposed an additional bug where we wouldn't look at the treemanifests
during the file computation, so when sending treeonly infinitepush bundles we
would get an error.
Reviewed By: singhsrb
Differential Revision: D7240340
fbshipit-source-id: a9be69597f4f1cbbecdd7cb1661f1023114bf621
Summary:
Sometimes infinitepush has to rebundle the data it's trying to send to
the client (like if you pull only part of a bundle). Previously, this was broken
for treeonly bundles because we did not correctly extend the bundlerepo's
data/history store on treemanifest servers. This patch fixes that and updates a
test.
Reviewed By: quark-zju
Differential Revision: D7240343
fbshipit-source-id: a9dd3ae884ace3fa9f4a748fe753fc394e69d6c9
Summary:
If we're pulling just part of an infinitepush bundle, it needs to
rebundle the data. We had a test for this for flat manifests, but we need a test
for treeonly commits as well. The test currently fails with an error. A future
diff will fix the error.
Reviewed By: quark-zju
Differential Revision: D7240342
fbshipit-source-id: 6acda3dfa6059f0957507a951df1b5e97df6dc4e
Summary:
Previously we just verified the pull worked. Let's also verify the
manifest is readable.
Reviewed By: quark-zju
Differential Revision: D7240341
fbshipit-source-id: 75c97591592699ff582fec5169ea20e0549fc603
Summary:
For short hashes it is nice to warn if the commit is too old.
Scenario:
After making commit on one machine, you can wait not enough time to
try to upload it on another machine using short hash from `hg sl`
in case of conflict (single conflict!) you can get automatically a very old commit,
that is not yours, without a warning. (And there were cases, when this screwed up hg that tried to go 250,000 commits ago)
I added a warning with a description of found commit if it is more than configured number of days old (default is 60)
Also, format of output has been redesigned. Now it is bold and got colors!
Also, I added #commitcloud
Another new features is that I added sorting by date in suggestions block.
Reviewed By: mjpieters
Differential Revision: D7195265
fbshipit-source-id: 7225a9b0fa8a7381f1e88ef7eb8a8fe2dfff43cd
Summary:
Once all clients are in treeonly mode, we can make the server tree only
mode as well and stop producing flat manifests entirely. This will speed up
pushes, save space, etc. This patch adds support for that mode.
Reviewed By: ryanmce
Differential Revision: D7148825
fbshipit-source-id: 15c899a58dd45af0c030c887c53f66347bc32b01
Summary:
The cstore doesn't contain our ondemand generator, nor our mutable data
store, so it's really not usable yet. Let's disable it for now until we can fix
up the issues that prevent us from using it (like having it report metrics).
Reviewed By: ryanmce
Differential Revision: D7148823
fbshipit-source-id: 5cc46af33c049b751c1c04916aafe0768d80ce7a
Summary:
Once we move to a treeonly world, there won't be any more manifest revlogs.
During the transition we'll still use hybrid manifest's though (the class, and
it will just resolve to treemanifests every time), so we need to support hybrid
manifests that don't actually have a revlog.
This also sets manifestlog._opener. We were already storing that on the other
manifestlog implementations, so we should do it on the main one for consistency
(and because this diff depends on it).
Reviewed By: ryanmce
Differential Revision: D7148821
fbshipit-source-id: 8f278a7bf4c0909423257165df59eae400a05a9e
Summary:
Previously the server would reject pushes that didn't use pushrebase,
since we relied on pushrebase to recreate the commits with the treemanifests.
Now that the flat-to-tree conversion logic is cleaner, we can use it to do the
conversion during a normal push.
Reviewed By: quark-zju
Differential Revision: D7143453
fbshipit-source-id: c13713e2ff59f1dff3ee1f44b7f8db7a92cfe1de
Summary:
This adds `progress.bar`, which is a context manager that provides a
time-updated progress bar, analogous to `progress.spinner`.
Reviewed By: ryanmce
Differential Revision: D7211895
fbshipit-source-id: be07d363f10c09164e2324d31a71a7c795257c8a
Summary:
Previously, in treeonly mode we would ignore any flat manifests that
were received in changegroups (via bundles or pulls, etc). This ended up causing
data loss in practice when people applied old bundles from before the
treemanifest conversion. Instead of just dropping those manifests, let's convert
them on the fly. This may be expensive, but it's better than losing the data.
A future diff may add caching to reuse flat text to speed up applying multiple
deltas.
Reviewed By: quark-zju
Differential Revision: D7083038
fbshipit-source-id: d2e350325d7e9005c8ddd5462034040274f790ff
Summary:
It makes sure that adding new options to the commands won't break them. For
example, rage sparse output was broken, and this diff fixes it.
Note that this changes behavior of the rage - if, say, smartlog extension is not enabled on the client, then there will be no output in the rage. Previously it wasn't the case. I think that's not the big problem, because:
a) The extensions are the most common ones and enabled everywhere
b) If they are disabled, then we'll immediately see a problem in hg rage.
However I need to change a test to make it pass. I need to add extensions and change grep output, because `hg rage --preview` has a few lines with `blackbox` in it.
Differential Revision: D7193297
fbshipit-source-id: dde2752ebc7dd3e3edea5c44576d0986f7d18744
Summary: sed -i without arguments doesn't work on OSX.
Reviewed By: farnz
Differential Revision: D7195193
fbshipit-source-id: a8eead927c94404a37ce5df956de82d29bc1b6a8
Summary:
In D7001328 we've added a new feature that skips commtis if there are no
changes relative to the sparse checkout. Unfortunately that causes lots of
treepacks downloads and makes bisect unusable. Let's revert the change.
Reviewed By: ryanmce, farnz
Differential Revision: D7182016
fbshipit-source-id: 274b29ca6a7b4c3faf83883b64f5ad3b0289873e
Summary:
Before this change `hg pullbackup` did not set correct markers on commits.
This change make possible to see what changes already landed even when we are restoring repository from backup.
Before the change `fbclone` + `hg pullbackup` of repo with `C1` commit landed would result in:
```
o o C2
| |
o o C1
| /
|
o
```
after:
```
o o C2
| |
o x C1
| /
|
o
```
Reviewed By: StanislavGlebik
Differential Revision: D7032572
fbshipit-source-id: ffee3c7cc23c24a3df9a89c999c9dd2de226dbff
Summary:
Rewriting a set of commits where there are replacement relationship among the
commits do not have an optimal UX today. For example, `rebase -s A -d Z` or
`metaedit A` in the below graph. B1, B2, C will all be replaced. But the new B1
and B2 replacement won't have the B1 -> B2 relationship, and the "new B1"
appears to be revived.
```
o C
|
x B1 (amended as B2)
|
| o B2
|/
o A o Z
```
One solution is to avoid rebasing `obsolete()::`, as implemented in D7067121
for metaedit. That would result in
```
o C
|
x B1 (amended as B2) o new B2
| |
x A o new A
```
The stack of A, B1, C is forced to break into two parts. This is fine for
power users. But n00b users would wonder why C is left behind. Per discussion
with simpkins at an internal post about the metaedit case, we think a more
linear history is more user-friendly. That is:
```
o new C
|
x new B1 (amended as *new* B2)
|
| o new B2
|/
o new A
```
The stack stays in a same shape.
This diff implements the "copying obsmarkers" behavior at the "createmarkers"
level so everything using that API would get the feature for free, including
metaedit and rebase.
D7067121 is reverted since the new UX is preferred. The test added is for
`metaedit` command, changes to rebase will be added in a later patch.
Differential Revision: D7121487
fbshipit-source-id: fd3c8a96ab434b131fb86d9882ccbdff8f63f05e
Summary:
Recently we unified the client and server code paths a bit, which
can cause the treemanifest server to attempt to do a prefetch (which doesn't
make sense since it has no where to prefetch from). It ends up throwing an Abort
error about not having a remote server configured. The fix is to make the
prefetch path smarter about when it's run on the server and to throw a standard
MissingNodesError instead. That kind of error is already handled in the hybrid
repository case and we just eat it and server the flat manifests like normal.
Once we move to treeonly mode, that error handler will re-raise the exception so
real issues with missing nodes won't be hidden.
Reviewed By: phillco
Differential Revision: D7182283
fbshipit-source-id: 15ed6549d9d7da1fee0570e1fa10338545ed92b1
Summary:
There's a bug where infinitepush attempting to rebundle a bundle that
does not contain trees causes an exception because the server attempts to
prefetch those trees (which fails because there's no where to prefetch from).
This diff just adds a test for that case. The next diff will fix it.
Reviewed By: phillco
Differential Revision: D7182284
fbshipit-source-id: a3fbb576cf3318c81b18943e0f0d466aa65e54fb
Summary:
All of the repos that use commit cloud have remotenames extension enabled, so bookprevnode and pushbackbookmarks parameters are not used. Local bookmarks won't be updated after a push.
We remove "bookprevnode" and "pushbackbookmarks" and functions related to them.
Reviewed By: StanislavGlebik
Differential Revision: D7122411
fbshipit-source-id: 0c6b3bc3f41f5b03d4bb2bc297ae35d77c90fedf
Summary:
The feature that automatically converted flat manifests to trees is
dependent on the hg server not sending flat manifests to treeonly clients
(otherwise it's very, very slow). Since the server rpms got reverted, we need to
backout these changes until the server issues are fixed.
Reviewed By: farnz
Differential Revision: D7181025
fbshipit-source-id: 1e4aad04d15909a3ce4f69313419e50c14bc8c19
Summary:
Previously, in treeonly mode we would ignore any flat manifests that
were received in changegroups (via bundles or pulls, etc). This ended up causing
data loss in practice when people applied old bundles from before the
treemanifest conversion. Instead of just dropping those manifests, let's convert
them on the fly. This may be expensive, but it's better than losing the data.
A future diff may add caching to reuse flat text to speed up applying multiple
deltas.
Reviewed By: quark-zju
Differential Revision: D7083038
fbshipit-source-id: 4912ec5ea5097163cede00158df821f116d92c9b
Summary:
This diff adds a config option to tweak deltabase in changegroup. It has 3
options:
- Always null - always use "null" as delta base, effectively make
everything full text
- No external - delta bases cannot be a revision outside the changegroup
- Default - the current behavior: delta bases can be anything that client
thinks the server should have.
This gives Mononoke more time to bake delta related logic, as we can
choose "always null" first, then incrementally increase the complexity.
Reviewed By: phillco
Differential Revision: D7158585
fbshipit-source-id: 5f6d9a78d1108093e8d08b9f296568f4f7e7471b
Summary:
Currently if you push or pull a bunch of commits between peers we will
include all the trees as part of the push. If the source repo doesn't have all
the necessary trees, it will go to the server to get them. Since the other
machine can just as easily go to the server (and probably won't need most of
those trees anyways), lets just have the source client send all draft trees and
skip the public commits,
Reviewed By: phillco
Differential Revision: D7141623
fbshipit-source-id: 6d33ae9d4c9cc32bf6dfa76f733c87c06890d719
Summary:
A future diff is going to change what trees are sent during a peer to
peer push. Let's update this test so we can see the actual changes in the next
diff.
Reviewed By: singhsrb
Differential Revision: D7141626
fbshipit-source-id: 75e61e9c417d86c48ed1762d6ab67bd4204f67c7
Summary:
`-r` seems to be unneeded, as tests pass without it.
Also, it does not look like the regex itself uses anything not mentioned in `man re_format` on OSX, so we can just use the non-extended re.
Reviewed By: StanislavGlebik
Differential Revision: D7167503
fbshipit-source-id: 3c5c520e9bf2627523cabc771226fc37dc2e9171
Summary:
This echos the change in D7056650; there is no need to special-case
treemanifests here; delegation to the manifest .match method allows the
manifest to apply optimisations when available.
Differential Revision: D7100363
fbshipit-source-id: 66a35850a132f804efb407712d2e4db737c10cff
Summary:
The current code iterates over all files in the manifest, filtering against a prefix.
But a manifest supports using a matcher directly, and efficient implementations like the treemanifest will prune the tree to a much smaller subset rapidly based on the path in a matcher. Switching to using a matcher dramatically improves --cwd-list performance in fbsource, when treemanifests are available.
Reviewed By: quark-zju
Differential Revision: D7056650
fbshipit-source-id: 2bf62ea93680323a49c9282266118805881d7b02
Summary:
For large updates, the dirstate update can take a while. Let's show
progress so the user understands what is happening and how long to wait.
Reviewed By: quark-zju
Differential Revision: D6903133
fbshipit-source-id: f7f6c3c14e1d3221a383da4a6e311aa12a8d3a98
Summary:
Added passing BatchMode option to SSH call only when puchbackup runs in background.
Also fixed dummyssh in skipping options before hostname, added unittest.
Reviewed By: quark-zju
Differential Revision: D7119123
fbshipit-source-id: 2c8e66fee44cca5b23389cba8e21e3a0b237268e
Summary:
Smartlog is supposed to show the latest public ancestor of all draft commits,
however this doesn't always happen.
The reason is a boundary error in the test for finding public commits. If the
latest public ancestor is also the common ancestor (fairly normal), then it
will be excluded.
Reviewed By: quark-zju
Differential Revision: D7140139
fbshipit-source-id: 6999f7ad14f86653ebe4d4f6543b9c7533871cf2
Summary:
Let's switch to xdiff for its better diff quality and performance!
The test changes demonstrate xdiff's better diff quality.
Reviewed By: ryanmce
Differential Revision: D7135206
fbshipit-source-id: 1775df6fc0f763df074b4f52779835d6ef0f3a4e
Summary:
The next test is going to switch bdiff to xdiff. This diff adds related tests
so we can clearly how xdiff improves the diff quality.
It also solves [issue5091](https://bz.mercurial-scm.org/5091) because xdiff
will shift hunks up and down to group them together. So that was also added as
a test. Although in a more complex case where the hunks are separated by some
common lines (ex. "Y"), xdiff won't help either.
Reviewed By: ryanmce
Differential Revision: D7147444
fbshipit-source-id: 3605290b5dfdfc7b8b004b38c7f7ee9534915380
Summary: This change minimally addresses the issue that `debugrebuilddirstate` can crash if the dirstate file is very corrupt.
Reviewed By: markbt
Differential Revision: D7028370
fbshipit-source-id: 72fc7a2900a8bc1bb5f062454530b4fc4c806f09
Summary: They help making tests easier to write.
Reviewed By: phillco
Differential Revision: D7121645
fbshipit-source-id: 9c7181d45c4e28155eb68f355cf1c4cfc077d191
Summary:
`.t` tests have some highly repeatitive logics, like creating a repo, etc.
This patch adds a common shell file that defines frequently used functions.
For now, `newrepo` and `enable` are added. The latter can be used to
enable a feature (ex. obsstore), or an extension.
`test-fb-hgext-fbamend-next.t` and `test-fb-hgext-absorb.t` are migrated
to use the new shell functions.
Reviewed By: phillco
Differential Revision: D7121485
fbshipit-source-id: 167fcc20e4e30864199b6c5af0958b80bfb68817
Summary:
Don't think this is required or used anymore and reveals information
about the structure of our project if we open source our mercurial.
Reviewed By: quark-zju
Differential Revision: D7128203
fbshipit-source-id: 4cdfa008631d08321a4d5a1c8f18cef429c35077
Summary:
The breakage we had on branch importer was related to filetransaction trying to
close a file that didn't exist. We're still not sure why this happens yet, but
the workaround was to use mercurial's transaction and to force not using workers,
which is this change.
Differential Revision: D7108127
fbshipit-source-id: 71fa63824984bfb91de3b732166f7bae496187ad
Summary: dispatch.lazyaliasentry didn't work with slices i.e. lazyasliasentry[:2] failed. This diff fixes it
Reviewed By: farnz
Differential Revision: D7109779
fbshipit-source-id: c704cd44fea0944ae4be68df36d32df98b7fc09b
Summary:
Previously pushrebase would only send changegroups using the cg1
format. remotefilelog will soon require cg2 (and it results in better deltas
anyway), so let's change pushrebase to allow using cg2.
Initially it is off by default. We will change it to be on by default once the
server has been upgrade to to handle the received part.
Reviewed By: mjpieters
Differential Revision: D7108732
fbshipit-source-id: ff4ad3a3fc2801aec4876db30c8130ce743b2e6a
Summary:
When background prefetch is enabled, let's use the base parameter to
limit how many files we download. This makes the operation O(files changed), and
on windows results in a significant speed up.
Reviewed By: mjpieters
Differential Revision: D7108838
fbshipit-source-id: a46b8a7d897ee204b9a4c1f1c65d875dbd3e9bc7
Summary:
There is no need to strip path separators from the normalised path; normpath
will never leave any.
We also ensure that the path prefix we test ends in a path separator, to avoid
matching on sibling paths that happen to share a prefix.
Reviewed By: ryanmce
Differential Revision: D7056649
fbshipit-source-id: 10b78a78ba44fbc8d9c05fb7ffd0ffd1c1496a67
Summary:
Eventually all the clients using the tree manifest will operate in
treeonly mode. In treeonly mode, we only read from tree manifests and
therefore, manifest related operations would fail for commits which only have
flat manifest. Example of such commits are old draft commits which were created
before the existence of tree manifest. One way to resolve this is to
automatically convert any flat manifests we come across to tree manifests. This
commit achieves that same.
Differential Revision: D7083033
fbshipit-source-id: 092fc7852ffc6d1b4130b5a1fc8d9e124cef4fcb
Summary:
This makes the mutable history pack implement the history store read
api so we can add it to the union store and read the contents of things that
have been written but not yet committed.
The mutablehistorypack fileentries variable has been changed to contain a dict
instead of a list so we can access it quickly during reads. The list is from a
legacy requirement where we used to maintain the order that the writer wrote in.
We no longer do that (instead we topologically sort what they've given us), so
switching from a list to a dict should be fine.
Differential Revision: D7083036
fbshipit-source-id: ae511db60ab6432059714a2271c175dc9683b8e1
Summary:
Now that _writeclientmanifest is basically just calling
manifestlog.add, let's get rid of it.
Differential Revision: D7083029
fbshipit-source-id: eee18cefd5a6ae3d95bba58b419364fc9fdb15b3
Summary:
In a future diff we'll need the ability to modify the union store on
the fly, so let's add addstore and removestore apis.
Reviewed By: ryanmce
Differential Revision: D7051102
fbshipit-source-id: 901a50720bfdf4e5c59714d092830e65edccdfce
Summary:
The union content store
- iterates through all the stores it has until the current store has the
content.
- Or, it fails eventually if none of the stores have the content.
It does so by relying on the current store throwing a KeyError if it doesn't
have the content.
`remotetreedatastore` was throwing the MissingNodesError which means any
remaining stores after it would not even get a chance to look for the content.
This commit addresses the same.
Reviewed By: ryanmce
Differential Revision: D6867854
fbshipit-source-id: 784df195efcbe16f2e716968f3d93159afff6206
Summary:
The root issue is described in the new ``workingflags()`` function -- ``workingfilectx.flags()`` ought to read from the working copy, but instead reads from the working copy's p1.
Rather than force a big rewrite to adapt all the old code (which might be a waste if we end up rewriting the context classes), let's add a new function with the behavior we want, and callers can detect for it and call it.
simplemerge is the only callsite that needs this behavior, AFAICT.
Previous attempt: D6694292
Reviewed By: singhsrb
Differential Revision: D7083239
fbshipit-source-id: 6f30b877958d40106203e088fe724ab491b541b9
Summary:
Remove the test for the `search` attribute on the passed-in regexp in
`hghave.matchoutput`. It's not necessary, and causes new check-code errors.
Reviewed By: quark-zju
Differential Revision: D7098248
fbshipit-source-id: b5f18c0db3cfbb37c9f7e23e2cdfdef9cedd3f49
Summary:
Some of our internal hg builds include the Eden extension now. This was
causing the test-help.t test to fail since it included help output for the Eden
extension, but the test code did not expect this.
We unfortunately cannot update the test output to always expect help output for
Eden since the Eden extension is only included in buck-based builds.
Reviewed By: quark-zju, farnz
Differential Revision: D7063937
fbshipit-source-id: e503ddc6889e546b5333a8d9e3555097d689e24c
Summary:
Update run-tests.py to cache the results of hghave checks. Some of the
linematch code can end up calling _hghave() hundreds of times in a row with the
same check.
This is particularly likely to happen on test failures, where a line with an
hghave check does not match the expected output. In this case the same hghave
check will be re-run on each subsequent line of test output.
Reviewed By: quark-zju, farnz
Differential Revision: D7077677
fbshipit-source-id: 768a11df78924b75cde4c8b48c801b79194f7969
Summary:
Add a new `--only-files-in-revs` option to `hg diff`. This filters the files
that are considered down to the set of files that are touched in either of the
two revisions.
Reviewed By: ryanmce
Differential Revision: D7070102
fbshipit-source-id: b9140cd3a212af9554ef5b76785210c1dbba8ee0
Summary:
metaedit when metaediting a commit with obsolete & visible children did
autorebase all of them causing a divergence (which broke the restack)
Reviewed By: ryanmce
Differential Revision: D7067121
fbshipit-source-id: 679bc7acecde1cd648adcdddc9c8ac3ecc101208
Summary:
When compiling with optimization enabled gcc complains that result is not
immediately initialized when created inside a constexpr function.
Differential Revision: D7065250
fbshipit-source-id: 5a829e775ce482c8a5ec9d1cfe279005ff11bb7e
Summary: Including bookmarks in the crdump output
Reviewed By: sfbahr
Differential Revision: D7063784
fbshipit-source-id: 08d38d8a73a8448b3a184bfc3511cdaec01771fb
Summary:
We don't really support files larger than 2G with files larger than
4G remotefilelog crashes badly.
Reviewed By: markbt
Differential Revision: D7066855
fbshipit-source-id: 40cdebbe703a7b3f13ce84174bf6f96565e8c3b7
Summary:
If the user specifies a revision using `-r`, along with
`--since-last-arc-diff`, then show the differences for that revision, rather
than the working copy parent.
Differential Revision: D7068359
fbshipit-source-id: feffadeb45eed4e1d8bb9d8fc7040932ada31736
Summary:
The fbhistedit `exec` verb attempts to create an obsmarker if the working copy
parent is changed as a result of the command execution.
This isn't always valid to do: the command could have created a new commit
entirely, and the old working copy parent could be public, and thus not
obsoletable.
It's also not necessary. If the command that ran during the exec did anything
that obsoleted commits (e.g. amend), then it should have created its own
obsmarker.
Differential Revision: D7066784
fbshipit-source-id: 326396828a06936c88154cfec6705f7834527a5f
Summary:
Inserting a commit after a public commit causes histedit to fail
when it tries to write out the obsmarkers.
Reviewed By: ryanmce
Differential Revision: D6820504
fbshipit-source-id: 1f73bde65eb87485b0712d6a365b846fd00ad48c
Summary:
Let's try to be consistent and print relative path names during merge. Note
that it changes test output for mq and subrepos, but we don't use these
features often, so it should be fine.
Differential Revision: D7067050
fbshipit-source-id: 3aba66e67657a1b1b30fcf239c03abd605eb5d9a
Summary:
`hg debugpathcomplete` is another example where the current implementation
iterates the whole dirstate looking for information that's in one directory.
With treedirstate, this can be made faster by using the tree structure.
Differential Revision: D7024784
fbshipit-source-id: 3c0b403887e85157b7eb2752abce5a0805e6ed1c
Summary:
"hg" only ignored ENOENT errors when including files (so it didn't abort when an included file didn't exist). But it aborted on every other error (e.g EPERM).
This diff makes it ignore any IOError and just print a warning.
Reviewed By: quark-zju
Differential Revision: D7029320
fbshipit-source-id: b1b8137cd575fc89fb5967e8d18fe82ab84a8f85
Summary:
Running `hg log --traceback --template '{file_copies}' -r XXXX` on a file with long history is slow for 2 reasons
- p4 fast importer preserves full history for deleted and re-added files
- p4 fast importer records the wrong parent of a file
This diff tries to fix these two issues.
In mercurial, if a file is added, deleted, and then added back, it should start a new file history when the file is added again.
For example,
commits commit1 commit2 commit3
actions add a.txt delete a.txt add a.txt
timeline ------------X------------X------------------------X------------
`hg debugindex a.txt` at commit3 shows a.txt as a new file without previous history
rev offset length delta linkrev nodeid p1 p2
0 0 3 -1 0 b789fdd96dc2 000000000000 000000000000
However, this is different in p4. `p4 filelog test.txt` gives you
//depot/Software/Apps/Main/Native/.castle/test.txt
... #3 change 523261 add on 2018/01/23 by zhihuih@devbig415 (text) 'test:add-again-same-file'
... #2 change 523254 delete on 2018/01/23 by zhihuih@devbig415 (text) 'testfile:delete'
... #1 change 523253 add on 2018/01/23 by zhihuih@devbig415 (text) 'testfile:add'
Currently, p4 fast importer preserves history the same way as p4, and this causes slowness (even timeout) in hg when it runs `hg log --traceback --template '{file_copies}' -r XXXX` on a revision that contains files with long history in p4 (mostly contributed by automation). To mitigate this, we want the p4 fast importer to behave the same way as hg, and starts a new history for a file that's added again.
Currently, p4 fast importer takes the tip of a filelog and treats that as the parent of the newly written entry diffusion/FBS/browse/master/fbcode/scm/hg/hgext/p4fastimport/importer.py;19ad9b05f50e3ff0265cdc7b4b45174dcf820343$468-469. This can be wrong when there are revisions from branches.
For example, if I edit file a in master in CL1, 2, 4, and I branch at CL3, and edit the file in branch in CL5, the current importer implementation will take filenode at CL4 as the parent of CL3
(CL1,2,3,4,5 corresponds to rev0,1,3,2,4)
{F120393661}
However, the correct behavior is to take filenode at CL2 as the parent of CL3
(CL1,2,3,4,5 corresponds to rev0,1,3,2,4)
{F120393662}
(This is also the example I use in `test-fb-hgext-p4fastimport-import-branch-filelogorder.t`, so if the description here looks confusing, please refer to the test)
Reviewed By: dsp
Differential Revision: D6962019
fbshipit-source-id: 24de76ae009e0d6f976d247087fe4702c99e0f82
Summary:
Update the treemanifest extension to also add a 'gettreepack()' function to
localpeer, and add the 'gettreepack' capability to localpeer if the repository
supports acting as a treemanifest server.
Also update the _addmanifestgroup() wrapper to only forbid it from being called
in a treemanifest server repository. When pulling from a local treemanifest
server repository this function may need to be invoked in the client
repository.
Reviewed By: DurhamG
Differential Revision: D6980931
fbshipit-source-id: 61519c7deeaa0347650323a0002b89597e91d9cc
Summary:
- Add support for RocksDB engine (developed as a drop in replacement for innodb) to hgsql to allow new xdb.hgsql.1-10 shards to host hg repos
- Prefer MySQL test DBs in same region
- Run all hgsql unit tests also for RocksDB engine
- Allow for nested ifs to make that possible (downside if you switch off rockdb tests, innodb tests are run twice)
Reviewed By: quark-zju
Differential Revision: D7014064
fbshipit-source-id: 073c36176aa7eaf74252ef33c3f47da594920b28
Summary: Add some unit tests for the functions in bitmanipulation.h
Reviewed By: quark-zju
Differential Revision: D7018862
fbshipit-source-id: 2ffbb0c6776f702ae1e5f9a19bd6646d1d576af0
Summary: Fixes T25823569. We need to check for mergestate last since some commands leave that file in addition to their own.
Reviewed By: DurhamG
Differential Revision: D7013846
fbshipit-source-id: 42730bbee81b4c3b3aa2280a48d57da703fab5dd
Summary:
Per discussion with @[100000771202578:kulshrax] [1], the new behavior is enabled everywhere, and is also vastly preferable. Let's nuke the old code, as well as the config declaration and all places that manually enabled it.
[1]
> also, it looks like the pre-restack logic is still in that function, and the modern restack code path is still gated by a config option
> we should probably remove that
Reviewed By: kulshrax
Differential Revision: D7031365
fbshipit-source-id: dcc620c292bac03fdc0aa144546d4628bb673046
Summary:
The treemanifest extension now uses a pattern of
treemod._cansendtrees() to detect if it should send trees. pushrebase and
infinitepush were using a different pattern where they were sending trees if
they existed (in infinitepush's case it was ignoring the sendtrees config
entirely). Let's make these consistent.
The new behavior requires that trees exist for the commits we're trying to send.
In order for this to work, we'll need the logic that automatically converts a
flat manifest into a treemanifest so that we're never missing a tree we want to
send.
Reviewed By: markbt
Differential Revision: D7004982
fbshipit-source-id: 7dd6788173a0393d7a129123d5d62256d64b8a64
Summary:
I will be enabling this config on its own more broadly soon. Let's add
a test just to verify it works without treeonly=True being set (which is what
most other tests already test).
Reviewed By: quark-zju
Differential Revision: D7004366
fbshipit-source-id: 5743fbfd23da7327d1779f8555c42a8845d60271
Summary:
There are cases when a local repo communicates to another local repo
where one of the repos has remotenames enabled and the other does not that lead
to crashes. We fix these cases by checking repo objects for _remotenames before
accessing that member variables.
Differential Revision: D7023300
fbshipit-source-id: e4df06671bfb8291f50c01b53750ccaa3f9df968
Summary:
Cases from sandcastle make it clear that there are circumstances where
a one local repo talks to another local repo, one with remotenames enabled and
the other without remotenames enabled, that crashes can happen.
Let's add a test case to find some of these so we can prove they are fixed.
Differential Revision: D7023299
fbshipit-source-id: ddc8a701ac7573bcc15f737df1c5e29ef3581201
Summary:
Previously, if the user had an svn or git push target set up, we would
bail from the normal push --to process even if the users were pushing to a
different location. This is silly. Instead, let's only bail if the user is
actually pushing to an svn or git destination repository.
This bug was discovered while testing the updated svn-to-hg sync scripts.
Differential Revision: D7023173
fbshipit-source-id: bec5e069bc043c2f73fb334bdaeb18a6fad93eef
Summary:
Today, there are cases where this function will be called but bookmarks
will not have been populated. We will eliminate one of these paths next,
but for safety, let's make sure we abort gracefully rather than with a
stacktrace by referencing something that does not exist.
Differential Revision: D7023172
fbshipit-source-id: 8e3b2afe9fe3b9d74d42aa5b947229418ec8c3ad
Summary:
When hg addremove needs to remove a file, remotefilelog tries to create a filenode in order to prefetch the removed files from the server. If the file is not in the parent context manifest, this throws an exception.
To solve this we have to first check if the file exists or not in the parent manifest. If the file does not exist in the manifest then we don't need to prefetch it, and addremove will behave like forget rather than remove.
Reviewed By: markbt
Differential Revision: D7009649
fbshipit-source-id: 0570bc00db546a455b9c2e4628740e24ca819dd6
Summary:
Upon running `p4fastimport` tests locally I found out this test being deleted was not working.
It turns out I was using a newer version of p4d (2017.2) and the clowny behavior
around symlinks that it tests for was fixed in p4d 2017.1
I found out we use 2017.1 in our perforce servers:
diffusion/O/browse/opsfiles/branches/PROD/chef/cookbooks/other/fb_perforce/recipes/perforce_server.rb;1001845$125,125-126,333-334
Therefore, this test is not currently accurate or needed
Differential Revision: D7015278
fbshipit-source-id: 6e0e62371781a06bf194a624f4f557bddd3ccda4
Summary:
Within buck test envirionment there is no guarnatee that `pyflakes` binary is
in `PATH`. This patch builds pyflakes as a test dependency so buck test will
run `test-check-pyflakes.t` instead of skipping it.
Reviewed By: markbt
Differential Revision: D7008100
fbshipit-source-id: eca6f0174dbc9c12d45eca2f17f70a9805f13094
Summary:
Currently, commits adding LFS files are not verified server-side since the LFS
extension explicitly skips hash checking during unbundle, to avoid overhead
downloading/reading LFS objects.
We'd like those commit hashes to be verified. However, the existing verify
command does not scale with a huge repo. So let's add `verify -r REV` support
to be able to incrementally verify a repo.
Reviewed By: quark-zju
Differential Revision: D6947227
fbshipit-source-id: 6ffda3a814822942630339fc7de62c3fcb284fda
Summary:
Previously we would only do ondemand downloading of tree blobs when
accessing data contents. When accessing just history we would just fail if the
data wasn't available locally. This adds the remote store to the history union
store so we can get history remotely.
Reviewed By: singhsrb
Differential Revision: D7003434
fbshipit-source-id: 839f8e84be35779ccb146d13ce3e1d6d1e7f46bd
Summary:
This can be convenient when the size of LFS downloads between r1 and r2 is needed.
NB: this does not pay attention to sparse profiles and acts as if the checkout was full. We probably need to know how to do both.
Reviewed By: quark-zju
Differential Revision: D6938259
fbshipit-source-id: 52ab88be83339472f2eccafc746a191ff26c16c7
Summary:
remotefilelog prefetch was broken in treeonly mode since it referred to
the manifest revlog to get the parents. Let's switch to the more modern
manifestctx way of accessing parent information.
Reviewed By: quark-zju
Differential Revision: D6995267
fbshipit-source-id: e0c11fca0f2156be3f936a6e437e7a4d3dffe75b
Summary:
quark-zju removed mangling of sys.path in D6974954. The `import sys` is now no
longer needed.
Differential Revision: D6998711
fbshipit-source-id: 1d8298ce8d5077e3ae7ae3c88f2a2b7adf9712d3
Summary:
It was used to patch Python's `SocketServer`. Since we no longer use
`SocketServer` in command server's code path. The extension can be removed.
Reviewed By: ryanmce
Differential Revision: D6974976
fbshipit-source-id: 52353b55d812e494756833cefba896b6e64eaf9e
Summary:
Make `library.sh` read `$HGTEST_GETDB_PATH` for `getdb.sh` location. Unify
`getdb.sh` used by infinitepush and hgsql into one.
Reviewed By: ryanmce
Differential Revision: D6974635
fbshipit-source-id: 55bcebfbdaca7f43fb0cfca0d3e089e76cf9f24e
Summary:
Mangling `sys.path` in `.py` tests could break tests running with `buck` opt
build. Because the bundled extensions won't be used, but files in the working
copy will. It's okay-ish for pure Python extensions. But native extensions will
have problems.
Reviewed By: ryanmce
Differential Revision: D6974954
fbshipit-source-id: 8a7f3d86e5c2fbf16cfcc971743d99585fe61b99
Summary:
anydbm uses bsddb by default. bsddb is slower than gdbm and
is not available in buck build. Therefore use gdbm instead.
Differential Revision: D6994367
fbshipit-source-id: 948b991df31436d02f478d7a95f9924e1f8e0ed5
Summary:
As part of producing only one pack file per transaction, we need to
change the mutabledatapack store to allow reads of written-but-not-finalized
data. This will allow us to add the mutabledatapack to the union store, and then
keep it alive for the duration of the transaction.
Reviewed By: quark-zju
Differential Revision: D6944348
fbshipit-source-id: 1e721bd8e07335a9c1f9c6b7595a765ec018c007
Summary:
Let's create a new portability header, which can be used on both Windows and
Posix.
Reviewed By: quark-zju
Differential Revision: D6970928
fbshipit-source-id: a3970c50260f52bfc0a9420a4ff11d93ace304b0
Summary:
We have seen several cases that 00changelog.i got filled with 0s at the end
after a hard reboot. Improve fixcorrupt tool to be able to detect and fix
that. Previously it will not treat node=nullid as corrupted, and it would
also trust linkrev=0, which will truncate the entire changelog.
Reviewed By: ryanmce
Differential Revision: D6952921
fbshipit-source-id: b77e85f23c304ebde3740d6ed70effa1bc792928
Summary:
`test-hgsql-verify.t` occassionally fails, however the error output is lost to
an invocation of grep.
Instead, capture the output of the call to `hg sqlverify`, and if it doesn't
contain the string we're looking for, print the whole output.
Reviewed By: DurhamG
Differential Revision: D6965336
fbshipit-source-id: 72c47bbafcdc2b3ba32dc307670f2d8b0ee09d16
Summary:
inhibit has a very hacky way to disable obsmarker cycle check.
Move that to core to clean up the code.
Reviewed By: ryanmce
Differential Revision: D6956217
fbshipit-source-id: 18370721ec80e74008232e09049a057ee080e7d5
Summary:
This check is useful and detects real errors (ex. fbconduit). Unfortunately
`arc lint` will run it with both py2 and py3 so a lot of py2 builtins will
still be warned.
I didn't find a clean way to disable py3 check. So this diff tries to fix them.
For `xrange`, the change was done by a script:
```
import sys
import redbaron
headertypes = {'comment', 'endl', 'from_import', 'import', 'string',
'assignment', 'atomtrailers'}
xrangefix = '''try:
xrange(0)
except NameError:
xrange = range
'''
def isxrange(x):
try:
return x[0].value == 'xrange'
except Exception:
return False
def main(argv):
for i, path in enumerate(argv):
print('(%d/%d) scanning %s' % (i + 1, len(argv), path))
content = open(path).read()
try:
red = redbaron.RedBaron(content)
except Exception:
print(' warning: failed to parse')
continue
hasxrange = red.find('atomtrailersnode', value=isxrange)
hasxrangefix = 'xrange = range' in content
if hasxrangefix or not hasxrange:
print(' no need to change')
continue
# find a place to insert the compatibility statement
changed = False
for node in red:
if node.type in headertypes:
continue
# node.insert_before is an easier API, but it has bugs changing
# other "finally" and "except" positions. So do the insert
# manually.
# # node.insert_before(xrangefix)
line = node.absolute_bounding_box.top_left.line - 1
lines = content.splitlines(1)
content = ''.join(lines[:line]) + xrangefix + ''.join(lines[line:])
changed = True
break
if changed:
# "content" is faster than "red.dumps()"
open(path, 'w').write(content)
print(' updated')
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
```
For other py2 builtins that do not have a py3 equivalent, some `# noqa`
were added as a workaround for now.
Reviewed By: DurhamG
Differential Revision: D6934535
fbshipit-source-id: 546b62830af144bc8b46788d2e0fd00496838939
Summary:
Solves issues below:
```
hgext/backups.py:18:1: F811 redefinition of unused 'registrar' from line 17
hgext/catnotate.py:1:1: F811 redefinition of unused 'util' from line 1
hgext/remotenames.py:57:5: F811 redefinition of unused 'registrar' from line 34
hgsubversion/setup.py:103:5: F401 'mercurial' imported but unused
hgsubversion/setup.py:109:5: F401 'hgsubversion.svnwrap.svn_swig_wrapper' imported but unused
i18n/polib.py:1281:29: F841 local variable 'exc' is assigned to but never used (Python 2)
i18n/polib.py:1427:13: F841 local variable 'typ' is assigned to but never used
i18n/polib.py:28:1: F401 'sys' imported but unused
mercurial/manifest.py:411:5: F811 redefinition of unused '_lazymanifest' from line 168
mercurial/posix.py:419:5: F811 redefinition of unused 'normcasefallback' from line 362
mercurial/posix.py:425:5: F811 redefinition of unused 'checkexec' from line 167
mercurial/posix.py:431:5: F811 redefinition of unused 'checklink' from line 234
mercurial/pycompat.py:29:5: F401 'http.cookiejar as cookielib' imported but unused
mercurial/pycompat.py:30:5: F401 'http.client as httplib' imported but unused
mercurial/pycompat.py:31:5: F401 'pickle' imported but unused
mercurial/pycompat.py:33:5: F401 'socketserver' imported but unused
mercurial/pycompat.py:34:5: F401 'xmlrpc.client as xmlrpclib' imported but unused
mercurial/statprof.py:573:36: F812 list comprehension redefines 'parent' from line 562 (Python 2)
mercurial/util.py:1076:5: F811 redefinition of unused 'nogc' from line 1051
mercurial/util.py:3221:5: F811 redefinition of unused 'dirs' from line 3184
tests/silenttestrunner.py:24:5: F811 redefinition of unused 'main' from line 6
tests/test-context.py:90:1: F811 redefinition of unused 'scmutil' from line 4
tests/test-fb-hgext-cstore-treemanifest.py:146:5: F811 redefinition of unused 'testDeeplyNested' from line 134
tests/test-fb-hgext-extutil.py:46:5: F811 redefinition of unused 'testbgcommandfailure' from line 37
tests/test_hgsubversion_util.py:47:1: F811 redefinition of unused 'svnwrap' from line 31 (Python 2)
tests/test_hgsubversion_util.py:49:1: F811 redefinition of unused 'svnwrap' from line 47 (Python 2)
```
Reviewed By: ryanmce
Differential Revision: D6934533
fbshipit-source-id: 8b51851a76fec88bb59107ed05a901d42c7326f8
Summary:
When running with a Python runtime with a slightly different zlib module,
some `zlib.compress` outputs are different. Some tests are testing the
length, or the content of `zlib.compress` output, directly or indirectly.
That's causing issues.
This patch adds a `common-zlib` hghave test so it can be used to gate tests
checking zlib output. Some lengths are also changed to glob patterns to be
compatible.
Reviewed By: ryanmce
Differential Revision: D6937735
fbshipit-source-id: 2328a39d7f2022f16d51f61b6178568b26dfe2fb
Summary:
This is similar to what D6925398 does. But covers areas that D6925398 missed
because the codemod script wasn't able to handle multiple-line `hg serve`
commands.
Reviewed By: DurhamG
Differential Revision: D6937919
fbshipit-source-id: a67de178527c11a0ed8bbac82f0c46d44b81be77
Summary:
`test-check-commit` gives false positives for some commits. Remove it.
Facebook
The commits that give false positives include commits that are outside of
`fbcode/scm/hg`.
Reviewed By: quark-zju
Differential Revision: D6819543
fbshipit-source-id: ddfaae7350d4ad6503b7a7ec22e899bb7ae743df
Summary:
Should be part of D6928820. This was not noticed since contrib/* and .t
changes did not trigger buck test.
Reviewed By: DurhamG
Differential Revision: D6933803
fbshipit-source-id: 0e7465f2440d2346418a1bf6ea21732a71cdf79f
Summary:
Previously we weren't able to commit right after creating a treeonly
repository. This was caused by the code attempting to read the null tree from
the store, which doesn't exist. The fix is to handle the null case and return an
empty tree instead of trying to look it up in the store. We already do this in a
number of other cases, so this was just a missing one.
Reviewed By: singhsrb
Differential Revision: D6930919
fbshipit-source-id: e227612be2640282eb997f4d563102d86f0be43a
Summary:
Port the double blank line and "foobar" naming rules to check-code.
check-commit checks "changes" while check-code checks "snapshot".
Checking changes would have a couple of disadvantages:
- No check after code is committed
- Could have difficulity dealing with commits involving other changes
in a mono repo.
Facebook
Context: https://fb.facebook.com/groups/scm/permalink/1558174000898881
Reviewed By: DurhamG
Differential Revision: D6928820
fbshipit-source-id: 9ea998731778150b60112840bce702d9584ba15b
Summary: This makes the test pass using a build without zstd.
Reviewed By: DurhamG
Differential Revision: D6928821
fbshipit-source-id: 68e858aa2d4e86c0271cd339385cc49179d97d92
Summary:
Previously `hg server` uses `HGPORT` that might be in use. This patch uses
`-p 0 --port-file ...` so `hg server` always gets assigned a free port.
The change was first made by the following Ruby script:
```
re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/
Dir['*.t'].each do |path|
old = File.read(path)
new = old.lines.map do |l|
next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/]
"#{l.chomp} (glob)\n"
end.join.gsub re, <<-'EOS'.chomp
$ hg serve\1 -p 0 --port-file $TESTTMP/.port \3
$ \2=`cat $TESTTMP/.port`
$
EOS
File.write(path, new) if old != new
end
```
Then there are some manual changes:
run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since
it does not know the assigned value in tests.
test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use
explicit paths instead of relying on `.hgrc` since the test restarts the
server and `.hg/hgrc` having an outdated URL.
test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it.
Changed the order so the correct `$HGPORT` is written.
test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`.
Reviewed By: DurhamG
Differential Revision: D6925398
fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
Summary:
This allows us to specify `-p 0 --port-file X` to get the port assigned in
an atomic way. So there won't be "server failed to start" caused by race
conditions in tests.
Reviewed By: DurhamG
Differential Revision: D6925397
fbshipit-source-id: 5bbb61b7eb2695f0a673afdb0730d2a61827f8b3
Summary:
This was unneeded. We can just populate the dict directly.
Note that now we catch any errors in the expected data format gracefully
and no longer stack trace with bad inputs.
Reviewed By: DurhamG
Differential Revision: D6826455
fbshipit-source-id: adc9cc1fc4895f3c67b112d914f566601336ce3b
Summary:
This would cause us to return data outside of our contract, which
crashes in a worse, less useful way that if we catch things ourselves and
print useful debug information.
Reviewed By: quark-zju
Differential Revision: D6826454
fbshipit-source-id: 9cdb2987f762c98c6167ffcea03545bc46eb2119
Summary: These cause tracebacks; grepped out for readability.
Reviewed By: DurhamG
Differential Revision: D6826450
fbshipit-source-id: 4ea27f0f30cd9ab97710a234f5a451912ccc55c1
Summary:
This will allow us to expose bugs in the graphql client code and
write tests to prevent future regressions. Note that this exposes a KeyError
bug when the response is malformed.
Having finally, we have arrived at full graphql responses both from the real
system and from mocking. We can now write tests more confidently.
Reviewed By: DurhamG
Differential Revision: D6826452
fbshipit-source-id: 4fd246fbafb353ce0138289262cbdfd0e9e35229
Summary:
I do not know why all of these fields that we never checked were
mocked in the fist place.
Reviewed By: DurhamG
Differential Revision: D6826457
fbshipit-source-id: 78d336940b6146b61684c7789172298497c0b57f
Summary:
Previously, we mocked returns from the code that processed the
graphql data, but that code has some bugs that were therefore not testable.
Here we move the tests to return mock graphql data, so we can also test the
processing functions. In future commits, we will fix some of the possible
issues.
Reviewed By: DurhamG
Differential Revision: D6826453
fbshipit-source-id: 5a390723e48ccf2477295602dd85bd065c072bd3
Summary:
This reveals an issue where rawsize stored in revlog could
be wrong.
Reviewed By: DurhamG
Differential Revision: D6920597
fbshipit-source-id: c195d2613c06455204cc59497bfb97aa963c529a
Summary:
Check to see if the module documentation is None before trying to call
`splitlines()` on it.
Reviewed By: singhsrb
Differential Revision: D6919043
fbshipit-source-id: 93c458cde9643a3f1b2d6fe6eb56fa312ae2a192
Summary:
Previosuly, we would miss anything logged in the `ui.atexit`-registered
handler, since the actual logging would happen before those handlers were
called.
Reviewed By: quark-zju
Differential Revision: D6912321
fbshipit-source-id: 77600b7ae535b4da56fef1f92b51998de8e304e2
Summary:
The remotefilelog cgunpacker logic could enter an infinite loop
if a copyfrom file node uses its copyto as delta base.
This repros without LFS in test-lfs-bundle.t.
Reviewed By: DurhamG
Differential Revision: D6910079
fbshipit-source-id: 99fea316e77218cd4bc9ea6f5506779a3e4ab9a6
Summary: Otherwise the copy data will be lost when applying LFS bundles.
Reviewed By: DurhamG
Differential Revision: D6906207
fbshipit-source-id: bc94c6614f9d4b2a2b4c7f44f57de49bd54d6b49
Summary: This is similar to the previous patch, but applies to remotefilelog.
Reviewed By: DurhamG
Differential Revision: D6906206
fbshipit-source-id: 2a9a56a57544b5e4d892f77438b2faaadece73ee
Summary:
This happens when the client with LFS revisions applies a bundle
with a delta base pointing to an LFS revision stored in the repo.
Reviewed By: DurhamG
Differential Revision: D6906210
fbshipit-source-id: 8b47f8304f8ef5ae4b02d7239b680f70106a4d83
Summary:
This is similar to what we have done for changegroups, but for non-changegroup
(addrawrevision) case. This is needed to make sure the delta application code
path can assume deltas are always against vanilla (non-LFS) rawtext so the next
fix becomes possible.
Reviewed By: DurhamG
Differential Revision: D6906202
fbshipit-source-id: a7d62dfed4206d45b42299f1dabf013620ae52b3
Summary:
There is no way to distinguish whether a delta base is LFS or non-LFS.
If the delta is against LFS rawtext, and the client trying to apply it has the
base revision stored as fulltext, the delta (aka. bundle) will fail to apply.
This patch forbids using delta on LFS revisions.
Note: this does not solve the problem entirely. Since the problem could also be
a client with base file revision being LFS tries to apply a non-LFS delta
(bundle).
Reviewed By: DurhamG
Differential Revision: D6878326
fbshipit-source-id: 9c3951e4673b8de61aae73a51e1bfff422f38d0f
Summary:
This is similar to `debugdata`, but instead of taking a file revision (or
file node in remotefilelog's case), it takes a revset.
This is more useful practically, since the user would know commit hashes
easily but file nodes are hidden from the UI.
This is intended to make it easier to investigate LFS contents.
Reviewed By: DurhamG, ryanmce
Differential Revision: D6891770
fbshipit-source-id: 415da9b773c30830a48c09eda9f1854c416e3222
Summary:
This diff fixes hg pullbackup so it retrieves commits in the same order that there were pushed. This was caused by commits being stored in sets and dictionare which are by unorered types. These have been replaced by OrderedDict to mantain the order.
**Description of the task:**
Infinitepush is a mercurial extension that allows sharing commits easily. Among other things it also backs up all local commites that were made on devservers and laptops. That means that we always have up-to-date backup You can read more about it here - https://our.intern.facebook.com/intern/dex/mercurial-workflow/infinitepush/ .
Backup is triggered whenever mercurial transaction is triggered i.e. during commit, rebase or any other operation that changes mercurial repo. `hg pullbackup` is the way to pull backed up commits.
There is a problem. Let's say host A has this repo:
` o`
` |`
` o C1`
` | /`
` | C2`
` | /`
` o`
So we have two draft commits: C1 and C2. C2 is probably an older commit. But if we restore it on another host we can get another output:
` o`
` |`
` o C2`
` | /`
` | C1`
` | /`
` o `
This happens because `hg sl` orders commits by it's revision number in revlog - https://www.mercurial-scm.org/wiki/Revlog .
The point of the task is to make sure commits are ordered by their creation date.
Link to comment: http://www.facebook.com/groups/scm/permalink/1504906422892306/?comment_id=1505017576214524&reply_comment_id=1506426179406997
Reviewed By: StanislavGlebik
Differential Revision: D6884670
fbshipit-source-id: 3281dbc1e25e24662a4b6ba78b96b85d5bae78c9
Summary:
If the pympler memory profiler is available, let's allow using SIGUSR2
to dump memory information in live Mercurial processes.
Reviewed By: quark-zju
Differential Revision: D6905664
fbshipit-source-id: fbfa8e538d12a52cf6e4f405ff701f542888cd7f
