Summary:
The "merge" metadata is a boolean value that indicates whether a merge of
two branches are ongoing and could have potential conflicts.
Note: state-leave does not mean the merge conflict is resolved. There might
still be unresolved conflicts in the working copy.
Differential Revision: D7286169
fbshipit-source-id: 0948174eb9a9d7c28af64b8ade58dc509d49f747
Summary:
Add a new extension, `clienttelemetry`, which sends information about the
client to the server. This information is logged by the server, which can use
the `sampling` extension to store this information for later analysis.
Includes a few basic pieces of information by default, including a correlator
which can be used to correlate logs from the client and the server.
Reviewed By: mjpieters
Differential Revision: D7271508
fbshipit-source-id: f152fe177d8bc73179d3a2c4d0d070d468885e2a
Summary: This part is empty anyway, so there is no point in sending it.
Reviewed By: DurhamG
Differential Revision: D7271595
fbshipit-source-id: 53bdff76d63f925725b00be8d836ffa87bb32470
Summary:
On a large repo, this can take seconds. Let the user see progress
during this time.
Differential Revision: D7211460
fbshipit-source-id: bfa9ad658a1c190399f9e33de7ddfbe006d6cbfb
Summary:
During testing, I discovered that a ctrl-c while `hg sparse --refresh`
was waitign on the wlock would cause a traceback. This fixes that.
Differential Revision: D7211459
fbshipit-source-id: dde006208f688cc7e2509e822443dececd59c08d
Summary:
I somehow had a brain fart before landing my infinitepush phases diff
and replaced a hash with a rev number, even though rev numbers don't work with
infinitepush. This fixes that.
fbshipit-source-id: aa85614d40e173f6472cb9570af8510eb15ea353
Summary: We need to call None on the progress bar so it knows it's done.
Reviewed By: ryanmce
Differential Revision: D7267691
fbshipit-source-id: 81b19d945a44e8f46a15abdaf501ef6b5dea4ffc
Summary:
Add obsmarkers to the things that are synced.
For the LocalService, obsmarkers are stored base64-encoded using version 1 of
the obsmarker binary format. This is sufficient for unit tests.
Differential Revision: D7181399
fbshipit-source-id: 61377105986de561622a160134d20fdbd54bb88f
Summary:
Adds the initial skeleton of the `commitcloud` extension to Mercurial, which
will synchronise heads and bookmarks (and later, obsmarkers) with commit cloud.
Current limitations:
- only supports a stub version of the service that uses local files
- requires manual pushbackup to send commits to the server via infinitepush
- doesn't do anything with obsmarkers
Differential Revision: D7167715
fbshipit-source-id: 53443f3fb8dff0fe52257d7b20fc601dbe10c883
Summary: Put titles, if present, into a second 'column' in the plain output.
Differential Revision: D7258046
fbshipit-source-id: dedbb04ecbf0b2262c4c507683262e4b7bef8d70
Summary:
Profiles can now have a [metadata] section, with key-value pairs defining some metadata to be associated with the profile.
This supports multiline entries by starting a continuation line with some whitespace:
```
[metadata]
title = Some cool title
description = A longer description
will be joined with newlines
provided you start with a space
or a tab.
```
Differential Revision: D7257628
fbshipit-source-id: d6b09161d54cb6824f691d07caee83d0f8bd0a84
Summary: We regularly parse multiple profiles, through includes or when discovering. Be clear what profile file caused the issue.
Differential Revision: D7257636
fbshipit-source-id: 1203c13f6c6a4e4d1397ace818aee21b9db38550
Summary:
infinitepush manipulates what phases are sent back to the client. The
old behavior was that it would report every commit from the bundle to the client
as draft, even if the commit wasn't related to the heads being fetched and even
if the commit wasn't actually a root. The mercurial client logic expected these
values to actually be roots, and this caused some logic to act incorrectly and
mark commits unrelated to the pulled head public.
The fix is to only send roots, and only ancestors of the requested head.
Reviewed By: singhsrb
Differential Revision: D7265155
fbshipit-source-id: 709c3c807258049e44e493b0668f3034a0b11328
Summary:
There's a bug in infinitepush where it can cause client phases to get
screwed up. This affects pulls where multiple stacks were uploaded at once, then
only part of the stack is pulled. Let's add a test to expose the bug.
Reviewed By: singhsrb
Differential Revision: D7265104
fbshipit-source-id: ec1320d46ec865e8fbcf324e8d94737a5c111ef7
Summary:
- Include line numbers for error messages
- Handle sections via a mapping, for ease of extending
- Use identity to test for the excludes section, because even if the contents
matched we'd want this to be an error.
Reviewed By: quark-zju
Differential Revision: D7239581
fbshipit-source-id: 36cf79102ed01758b809009bcbe470714ddd27e4
Summary: This makes it easier to add metadata later on, and code that needs only a single entry becomes cleaner.
Reviewed By: quark-zju
Differential Revision: D7239580
fbshipit-source-id: 8900b4fd4883e68ac599396e4a80063d11bbf07c
Summary:
attrs is more flexible than a namedtuple, and we don't need indices on these objects.
The object is still immutable and has a low memory profile, but now we can trivially validate values. It's easier to extend the class now to add metadata.
Reviewed By: quark-zju
Differential Revision: D7239584
fbshipit-source-id: cbd0b38c54f08ed19d5a7e05811356e4694d7d19
Summary: Delete an unnecessary test command which delete a non-existed local bookmark.
Reviewed By: StanislavGlebik
Differential Revision: D7216466
fbshipit-source-id: f8fc4215ef1fa725c5e5f92ef03cc2616326ac59
Summary:
Previously, if we weren't sending trees we would not attempt to process
what files needed to be sent. This was incorrect, since files may be sent
independent of what trees are decided to be sent. So let's update the code to
care about shouldaddfilegroup instead of cansendtrees.
This exposed an additional bug where we wouldn't look at the treemanifests
during the file computation, so when sending treeonly infinitepush bundles we
would get an error.
Reviewed By: singhsrb
Differential Revision: D7240340
fbshipit-source-id: a9be69597f4f1cbbecdd7cb1661f1023114bf621
Summary:
Sometimes infinitepush has to rebundle the data it's trying to send to
the client (like if you pull only part of a bundle). Previously, this was broken
for treeonly bundles because we did not correctly extend the bundlerepo's
data/history store on treemanifest servers. This patch fixes that and updates a
test.
Reviewed By: quark-zju
Differential Revision: D7240343
fbshipit-source-id: a9dd3ae884ace3fa9f4a748fe753fc394e69d6c9
Summary:
If we're pulling just part of an infinitepush bundle, it needs to
rebundle the data. We had a test for this for flat manifests, but we need a test
for treeonly commits as well. The test currently fails with an error. A future
diff will fix the error.
Reviewed By: quark-zju
Differential Revision: D7240342
fbshipit-source-id: 6acda3dfa6059f0957507a951df1b5e97df6dc4e
Summary:
Previously we just verified the pull worked. Let's also verify the
manifest is readable.
Reviewed By: quark-zju
Differential Revision: D7240341
fbshipit-source-id: 75c97591592699ff582fec5169ea20e0549fc603
Summary:
For short hashes it is nice to warn if the commit is too old.
Scenario:
After making commit on one machine, you can wait not enough time to
try to upload it on another machine using short hash from `hg sl`
in case of conflict (single conflict!) you can get automatically a very old commit,
that is not yours, without a warning. (And there were cases, when this screwed up hg that tried to go 250,000 commits ago)
I added a warning with a description of found commit if it is more than configured number of days old (default is 60)
Also, format of output has been redesigned. Now it is bold and got colors!
Also, I added #commitcloud
Another new features is that I added sorting by date in suggestions block.
Reviewed By: mjpieters
Differential Revision: D7195265
fbshipit-source-id: 7225a9b0fa8a7381f1e88ef7eb8a8fe2dfff43cd
Summary:
Once all clients are in treeonly mode, we can make the server tree only
mode as well and stop producing flat manifests entirely. This will speed up
pushes, save space, etc. This patch adds support for that mode.
Reviewed By: ryanmce
Differential Revision: D7148825
fbshipit-source-id: 15c899a58dd45af0c030c887c53f66347bc32b01
Summary:
Every update to capabilities was requiring changing two spots. Let's
move that into a function. In a future diff we'll add another capability here.
Reviewed By: ryanmce
Differential Revision: D7148827
fbshipit-source-id: 5e8e95c330f2328c7e6c2b15ec708a45a72cf649
Summary:
Currently revlog writing is done through a custom external path. Let's
hide it behind manifestlog.add as well. This will make it simpler to support the
transition to a tree only server in an upcoming diff.
Reviewed By: ryanmce
Differential Revision: D7148822
fbshipit-source-id: af43a639395e4686cf4b3d654d4408e0747c1b47
Summary:
Now that all revlog access happens through the store, let's drop the
duplicate revlog instance on the treemanifestlog. Instead let's get the copy
from the store.
Reviewed By: ryanmce
Differential Revision: D7148819
fbshipit-source-id: 3194247de583c55400b52a3b292efa7b9f9a4649
Summary:
Previously, the hg server used the python treemanifest implementation
to represent and write treemanifests. As part of unifying the code base, and
making it easier to enable treeonly servers in a future patch, let's move all
tree server logic to use cstore treemanifest instead.
Reviewed By: ryanmce
Differential Revision: D7148826
fbshipit-source-id: 81003fbeef94e448ea3aca3380af1b4a9f8fd22d
Summary:
The cstore doesn't contain our ondemand generator, nor our mutable data
store, so it's really not usable yet. Let's disable it for now until we can fix
up the issues that prevent us from using it (like having it report metrics).
Reviewed By: ryanmce
Differential Revision: D7148823
fbshipit-source-id: 5cc46af33c049b751c1c04916aafe0768d80ce7a
Summary:
Once we move to a treeonly world, there won't be any more manifest revlogs.
During the transition we'll still use hybrid manifest's though (the class, and
it will just resolve to treemanifests every time), so we need to support hybrid
manifests that don't actually have a revlog.
This also sets manifestlog._opener. We were already storing that on the other
manifestlog implementations, so we should do it on the main one for consistency
(and because this diff depends on it).
Reviewed By: ryanmce
Differential Revision: D7148821
fbshipit-source-id: 8f278a7bf4c0909423257165df59eae400a05a9e
Summary:
In a future patch we will start using the contents of the
manifestrevlogstore as the place we call add() to add new revisions. In order to
do that, those revlogs must be manifestrevlogs. This patch just changes them to
use the right type.
Reviewed By: quark-zju
Differential Revision: D7148824
fbshipit-source-id: 0d60cd22b041db83b61ccaec3b6172624ea97e42
Summary:
Previously we passed the p1tree directly. It's actually more convient
to pass the p1node, since in a future diff we'll be adding data to the tree
revlogs and they need p1node. This is also safer, since now the caller can't
accidentally pass the wrong p1tree.
Reviewed By: quark-zju
Differential Revision: D7148820
fbshipit-source-id: a489308ebcb2960643b61ff73dc90f1285a94ae3
Summary:
Previously the server would reject pushes that didn't use pushrebase,
since we relied on pushrebase to recreate the commits with the treemanifests.
Now that the flat-to-tree conversion logic is cleaner, we can use it to do the
conversion during a normal push.
Reviewed By: quark-zju
Differential Revision: D7143453
fbshipit-source-id: c13713e2ff59f1dff3ee1f44b7f8db7a92cfe1de
Summary:
This adds `progress.bar`, which is a context manager that provides a
time-updated progress bar, analogous to `progress.spinner`.
Reviewed By: ryanmce
Differential Revision: D7211895
fbshipit-source-id: be07d363f10c09164e2324d31a71a7c795257c8a
Summary:
The normal `ui.progress` function is noticably slow due to the debug code, even
when debug is disable. Prevent this from happening by bypassing it completely
when debug is not enabled.
Reviewed By: ryanmce
Differential Revision: D7213855
fbshipit-source-id: ecea0e610ba01ad35c44cd7bd8cac74b8e00621d
Summary:
During profiling of the `hg sparse --refresh` command, I discovered
significant time spent in producing the progress bar. At first I was skeptical,
but then I commented out the progress line and the command took half as long!
I started looking for micro-optiization opportunities.
Differential Revision: D7210617
fbshipit-source-id: 17a11ae572fee48e6d8db33ca75843be0ec2e850
Summary: It can be useful for debugging why hg status is slow
Reviewed By: phillco
Differential Revision: D7212530
fbshipit-source-id: ee1b6979717604fd3cf9819748aa97a61eec0540
Summary:
xdiff has a `xdl_trim_ends` step that removes common lines, unmatchable
lines. That is in theory good, but happens too late - after splitting,
hashing, and adjusting the hash values so they are unique. Those splitting,
hashing and adjusting hash values steps could have noticeable overhead.
For not uncommon cases like diffing two large files with minor differences,
the raw performance of those preparation steps seriously matter. Even
allocating an O(N) array and storing line offsets to it is expensive.
Therefore my previous attempts [1] [2] cannot be good enough since they do
not remove the O(N) array assignment.
This patch adds a preprocessing step - `xdl_trim_files` that runs before
other preprocessing steps. It counts common prefix and suffix and lines in
them (needed for displaying line number), without doing anything else.
Testing with a crafted large (169MB) file, with minor change:
```
open('a','w').write(''.join('%s\n' % (i % 100000) for i in xrange(30000000) if i != 6000000))
open('b','w').write(''.join('%s\n' % (i % 100000) for i in xrange(30000000) if i != 6003000))
```
Running xdiff by a simple binary [3], this patch improves the xdiff perf by
more than 10x for the above case:
```
# xdiff before this patch
2.41s user 1.13s system 98% cpu 3.592 total
# xdiff after this patch
0.14s user 0.16s system 98% cpu 0.309 total
# gnu diffutils
0.12s user 0.15s system 98% cpu 0.272 total
# (best of 20 runs)
```
It's still slightly slower than GNU diffutils. But it's pretty close now.
Testing with real repo data:
For the whole repo, this patch makes xdiff 25% faster:
```
# hg perfbdiff --count 100 --alldata -c d334afc585e2 --blocks [--xdiff]
# xdiff, after
! wall 0.058861 comb 0.050000 user 0.050000 sys 0.000000 (best of 100)
# xdiff, before
! wall 0.077816 comb 0.080000 user 0.080000 sys 0.000000 (best of 91)
# bdiff
! wall 0.117473 comb 0.120000 user 0.120000 sys 0.000000 (best of 67)
```
For files that are long (ex. commands.py), the speedup is more than 3x, very
significant:
```
# hg perfbdiff --count 3000 --blocks commands.py.i 1 [--xdiff]
# xdiff, after
! wall 0.690583 comb 0.690000 user 0.690000 sys 0.000000 (best of 12)
# xdiff, before
! wall 2.240361 comb 2.210000 user 2.210000 sys 0.000000 (best of 4)
# bdiff
! wall 2.469852 comb 2.440000 user 2.440000 sys 0.000000 (best of 4)
```
The improvement is also seen for the `json` test case mentioned in D7124455.
xdiff's time improves from 0.3s to 0.04s, similar to GNU diffutils.
This patch is also sent as https://phab.mercurial-scm.org/D2686.
[1]: https://phab.mercurial-scm.org/D2631
[2]: https://phab.mercurial-scm.org/D2634
[3]:
```
// Code to run xdiff from command line. No proper error handling.
mmfile_t readfile(const char *path) {
struct stat st; int fd = open(path, O_RDONLY);
fstat(fd, &st); mmfile_t f = { malloc(st.st_size), st.st_size };
ensure(read(fd, f.ptr, st.st_size) == st.st_size); close(fd); return f; }
static int xdiff_outf(void *priv_, mmbuffer_t *mb, int nbuf) { int i;
for (i = 0; i < nbuf; i++) { write(STDOUT_FILENO, mb[i].ptr, mb[i].size); }
return 0; }
int main(int argc, char const *argv[]) {
mmfile_t a = readfile(argv[1]), b = readfile(argv[2]);
xpparam_t xpp = { XDF_INDENT_HEURISTIC, 0 };
xdemitconf_t xecfg = { 3, 0 }; xdemitcb_t ecb = { 0, &xdiff_outf };
xdl_diff(&a, &b, &xpp, &xecfg, &ecb); return 0; }
```
Reviewed By: ryanmce
Differential Revision: D7151582
fbshipit-source-id: 3f2dd43b74da118bd827af4fc5e1bf65be191ad2
Summary:
`mmap_readonly` will be reused in `index.rs` so let's moved it to a shared
utils module.
Reviewed By: DurhamG
Differential Revision: D7208359
fbshipit-source-id: d98779e4e21765ce0e185281c9560245b59b174c
Summary:
Add ScopedFileLock. This is similar to Python's contextmanager.
It's easier to use than the fs2 raw API, since it guarantees the file is
unlocked.
Reviewed By: jsgf
Differential Revision: D7203684
fbshipit-source-id: 5d7beed99ff992466ab7bf1fbea0353de4dfe4f9
Summary: They are simpler than radix entry and similar.
Reviewed By: DurhamG
Differential Revision: D7191652
fbshipit-source-id: b516663567267a2e354748396b44c2ac8ebb691f
Summary: These are Rust structures that map to the file format.
Reviewed By: DurhamG
Differential Revision: D7191366
fbshipit-source-id: 23a4431383be9713e955b74306cd68108eb80536
