Commit Graph

3526 Commits

Author SHA1 Message Date
Jun Wu
f629c0097d io: flush progress output
Summary: Make sure the progress is not hidden because of buffering.

Reviewed By: singhsrb

Differential Revision: D26886277

fbshipit-source-id: d44e48e67b4529a181dd36c30e92608654d8fea6
2021-03-10 19:36:59 -08:00
Jun Wu
2a4db136e5 mergetools: add new conflict marker format with diffs in
Summary:
Written by Martin von Zweigbergk at https://phab.mercurial-scm.org/D9551,
or https://www.mercurial-scm.org/repo/hg/rev/bdc2bf68f19e.
He suggested it and it's a useful feature. I did minor compatibility changes
(encoding, avoid rev numbers, foo_bar -> foobar).

Original commit message:

I use 3-way conflict markers. Often when I resolve them, I manually
compare one the base with one side and apply the differences to the
other side. That can be hard when the conflict marker is large. This
patch introduces a new type of conflict marker, which I'm hoping will
make it easier to resolve conflicts.

The new format uses `<<<<<<<` and `>>>>>>>` to open and close the
markers, just like our existing 2-way and 3-way conflict
markers. Instead of having 2 or 3 snapshots (left+right or
left+base+right), it has a sequence of diffs. A diff looks like this:

```
------- base
+++++++ left
 a
-b
+c
 d
```

A diff that adds one side ("diff from nothing") has a `=======` header
instead and does not have have `+` prefixed on its lines. A regular
3-way merge can be viewed as adding one side plus a diff between the
base and the other side. It thus has two ways of being represented,
depending on which side is being diffed:

```
<<<<<<<
======= left
contents
on
left
------- base
+++++++ right
 contents
 on
-left
+right
>>>>>>>
```
or
```
<<<<<<<
------- base
+++++++ left
 contents
 on
-right
+left
======= right
contents
on
right
>>>>>>>
```

I've made it so the new merge tool tries to pick a version that has
the most common lines (no difference in the example above).

I've called the new tool "mergediff" to stick to the convention of
starting with "merge" if the tool tries a regular 3-way merge.

The idea came from my pet VCS (placeholder name `jj`), which has
support for octopus merges and other ways of ending up with merges of
more than 3 versions. I wanted to be able to represent such conflicts
in the working copy and therefore thought of this format (although I
have not yet implemented it in my VCS). I then attended a meeting with
Larry McVoy, who said BitKeeper has an option (`bk smerge -g`) for
showing a similar format, which reminded me to actually attempt this
in Mercurial.

Reviewed By: DurhamG

Differential Revision: D26947920

fbshipit-source-id: 8b4446862897ff9a6dfdf5a2e35617d4db09e883
2021-03-10 17:29:18 -08:00
Jun Wu
16564f94c9 minibytes: into_vec cannot use fast path if the vec is sliced
Summary:
Add a check to ensure the `into_vec()` fast path do not take the full vec if
the `Bytes` is sliced.

Reviewed By: andll

Differential Revision: D26966453

fbshipit-source-id: 538dfb8ca2f01a46d1ede7b98b7f0a30fc7a786e
2021-03-10 17:16:53 -08:00
Durham Goode
9b3dacb521 fsmonitor: add fsmonitor.watchman-query-lock
Summary:
VS Code is seeing issues where they are accidentally triggering
multiple hg status calls, which trigger multiple expensive watchman queries.
While they're trying to track down why this is happening, they'd like a config
that would enable hg to only run one of their status calls at a time.

Reviewed By: quark-zju

Differential Revision: D26931193

fbshipit-source-id: 3b36ac06217bb506110b8d708d4a74378245d5bb
2021-03-10 17:07:08 -08:00
Jun Wu
cb39612472 visibility: extract pure deserialization to a function
Summary: Similar to D26924712 (1225d154d8). It'll be reused elsewhere.

Reviewed By: kulshrax

Differential Revision: D26935028

fbshipit-source-id: 26e1271a4fc61559a2f659d052a6adbf509eace3
2021-03-10 16:49:59 -08:00
Jun Wu
44a226e7e6 error: make BadResponseError also RepoError
Summary:
RepoError prints `!` at the end of `abort:`. This maintains the old behavior
and should fix test breakages like:

     $ hg clone 'ssh://fakehost|touch%20owned/path'
     destination directory: path
  -  abort: no suitable response from remote hg!
  +  abort: no suitable response from remote hg
     [255]

Reviewed By: DurhamG

Differential Revision: D26964146

fbshipit-source-id: 2e6d095b82ce1e2e23a353cf8f06ae844ee872d7
2021-03-10 16:01:52 -08:00
Jun Wu
33b28a117e smartlog: limit commits to show
Summary:
There are a report of slow smartlog that tries to show 8M nodes in `draft()`.
That is an issue after the automatic bookmark clean up, and the visible heads
incorrectly make large amount of commits draft.

Reviewed By: kulshrax

Differential Revision: D26934644

fbshipit-source-id: 0c0890f8eaf1422dab9c03159a419800ae7247ca
2021-03-10 15:24:39 -08:00
Jun Wu
a22c4883f3 context: make memctx.mirror copy mutinfo and loginfo
Summary: Those were previously ignored incorrectly.

Reviewed By: kulshrax

Differential Revision: D26957315

fbshipit-source-id: 5366460c1368ab9a83471d092bb860cc85fb64c3
2021-03-10 15:20:14 -08:00
Durham Goode
5276ded496 watchman: log when hg checkout triggers watchman recrawls
Summary:
We're seeing issues where watchman is frequently having to recrawl the
repository due to fsevents getting dropped. We believe this is caused by large
amounts of IO, but we're not sure from what process (buck? hg? arc? ide?). Let's
add some logging to Mercurial to estimate when a checkout triggers a recrawl.

Reviewed By: xavierd

Differential Revision: D26931996

fbshipit-source-id: 0026c792f0ec216cb3e98424da819c4c6e925072
2021-03-10 14:22:44 -08:00
Jun Wu
b1547cad1a bookmarks: clean up visibleheads if remotenames are removed
Summary:
Visible heads can overlap with scratch remote bookmarks, and in bad cases
(before D26792731 (7a759b6075)), overlap with public remotenames.

When we remove remotenames we need to remove referred nodes in visibleheads
too so we don't end up with massive draft commits.

Reviewed By: DurhamG

Differential Revision: D26954215

fbshipit-source-id: 91010e6608d0150ecf374ce31705e97712154b27
2021-03-10 11:55:53 -08:00
Jun Wu
3fdd766b65 bookmarks: clean up scratch remotenames
Summary:
Scratch remote bookmarks are considered draft() and were not cleaned up.
Practically there are users with 100+ scratch remote bookmarks. Let's
clean them up too.

Note the commit still remain visible, which will be fixed in upcoming
diffs.

Reviewed By: DurhamG

Differential Revision: D26954216

fbshipit-source-id: e84f99e4e914f0c5169583fc9f60084f23664e02
2021-03-10 11:55:53 -08:00
Andrey Chursin
a3b1e04c7e async_runtime: block_on_future to call block_on_exclusive
Summary:
Current `block_on_future` implementation is not safe, as it does not panic on nested calls. Nesting `block_on_future` calls causes starvation of tokio scheduler processes and eventually will lead to deadlock if nested too many times

This diff simply calls `block_on_exclusive` from `block_on_future`. When we see that this does not cause problems we can simply remove `block_on_future`

Reviewed By: sfilipco

Differential Revision: D26899522

fbshipit-source-id: 93794bf2c5908421691dfb094d1807266c9ecd8d
2021-03-10 11:20:45 -08:00
Jun Wu
ea9cac2520 fsync: add allheads and lfs to fsync list
Summary:
Johan Schuijt-Li found they are broken on some CI hosts.
(But other store paths seem okay, which might indicate the fsync list is effective)

Reviewed By: DurhamG

Differential Revision: D26950101

fbshipit-source-id: e3bcd3f77636325be9e9ce8dfded8b17ec68f436
2021-03-10 11:06:10 -08:00
Thomas Orozco
f5f78c4906 third-party/rust: update tokio & tokio-stream
Summary:
Pulling this for those 2 PRs:

- https://github.com/tokio-rs/tokio/pull/3547
- https://github.com/tokio-rs/tokio/pull/3576

Reviewed By: ahornby

Differential Revision: D26944216

fbshipit-source-id: ad67afa69cb291cfb1622cf4b2a10727a13d19cd
2021-03-10 11:01:28 -08:00
Jun Wu
cac01986ae debugreseteads: command to reset heads to a fresh state
Summary: This is intended to be used in Sandcastle / OnDemand use-cases to provide a cleaner state.

Reviewed By: DurhamG

Differential Revision: D26924711

fbshipit-source-id: a37d24a10c99d953e2af842f729ef634cbb2e2f9
2021-03-10 10:44:44 -08:00
Durham Goode
cf95f40455 py3: fix eol filemerge logic
Summary:
Some merge-tools, like kdiff3, are configured to fix end of line stuff.
This code breaks in Python 3.

Reviewed By: quark-zju

Differential Revision: D26932815

fbshipit-source-id: 7601d53616e50961b89bbc4b0340a9fb672293ef
2021-03-10 09:39:46 -08:00
Jun Wu
1225d154d8 visibility: extract pure serialization to a function
Summary: It'll be reused elsewhere.

Reviewed By: kulshrax

Differential Revision: D26924712

fbshipit-source-id: 0be666a1acd0bdfbc14bd39d4193a06a9929eed7
2021-03-09 18:52:24 -08:00
Jun Wu
3db2121dec async-runtime: add a way to spawn task using the runtime
Summary: This API will be used to spawn tasks in the runtime.

Reviewed By: andll

Differential Revision: D26615607

fbshipit-source-id: 6b9892fad755bbe8feb775e9dad457697b2ea1b7
2021-03-09 11:45:28 -08:00
Jun Wu
846050ea0f changelog: disable migrating backend for hgsql repos
Summary: hgsql can only use traditional revlog. Disable changelog migration for it.

Reviewed By: kulshrax

Differential Revision: D26891252

fbshipit-source-id: 36c5a448d4fcad15b3415e4534448a945f6d0b4b
2021-03-08 15:58:36 -08:00
Jun Wu
5b7b35d399 changelog: add missing changelog type logging
Summary: A few types weren't logged properly. Add them.

Reviewed By: kulshrax

Differential Revision: D26891253

fbshipit-source-id: 64f59c56663b4f395679307df4a75dcff1ff811f
2021-03-08 15:58:36 -08:00
Stefan Filip
95ab9679e2 error: fix constructor for NetworkError
Summary: hint is an expected valid keyword argument.

Reviewed By: quark-zju

Differential Revision: D26896943

fbshipit-source-id: 006413838fbfa63eccc24cbc98d0e713d5237a69
2021-03-08 14:56:26 -08:00
Arun Kulshreshtha
c94dfbcc38 clidispatch: pass repo path to config contructor in from_raw_path
Summary: We weren't passing a repo path when initially loading the repo's config in `clidispatch`. This meant that the resulting `ConfigSet` didn't contain values from the shared-repo `.hgrc.dynamic`. Evidently, some other code path in Python would eventually add these values, but this meant that pure-Rust commands could not see config values set via dynamicconfig. Passing the path fixes the problem.

Reviewed By: DurhamG

Differential Revision: D26508980

fbshipit-source-id: 65f187d18098a08c81325e78cb02a8ed854c739a
2021-03-08 13:31:10 -08:00
Jun Wu
02ff6e200c remotenames: clean up bookmarks that are ancestors of master
Summary:
See the previous diff for motivation. This removes bookmarks that are
ancestor of master, too. This is important in practice.

Reviewed By: DurhamG

Differential Revision: D26889412

fbshipit-source-id: 255722ed5b486e88ef56e7e378fae3f1113d5fbe
2021-03-08 11:28:40 -08:00
Jun Wu
2e353eb4ce tests: add a test showing release bookmarks aren't automatically cleaned up
Summary:
The auto cleanup was conservative. It keeps `::draft()`. But that means
ancestors of public commits are not cleaned up. Not all release branches
branch off the master branch.

Reviewed By: DurhamG

Differential Revision: D26889413

fbshipit-source-id: c6a8e3f32cf1f7d2ffe74b7ecd183f4e583949bb
2021-03-08 11:28:40 -08:00
Stefan Filip
f172c2f8ae mononokepeer: wrap read exceptions with NetworkError
Summary:
This allows for errors raised in these cases to be retried. Most notable is
the timeout error.

Reviewed By: johansglock

Differential Revision: D26855441

fbshipit-source-id: 6137ed1755072d43dbdd25fa092ddb21c8669aa3
2021-03-08 08:06:01 -08:00
Stefan Filip
573c25553a mononokepeer: add timeout to socket
Summary:
No timeout is set up by default so the process wait forever when reading bytes
in cases where the connection is lost somehow.

Reviewed By: johansglock

Differential Revision: D26855443

fbshipit-source-id: d741f73e7186fe862f3d78a806f3219c2cbe7e0a
2021-03-08 08:06:00 -08:00
Stefan Filip
882c8c70a7 mononokepeer: prefer NetworkError over Abort
Summary:
Abort one of the most general exceptions in Mercurial. In theory it should be
something that isn't handled. We can say at least that it shouldn't be retried.
For thing that may be transient it is better to use a different type of
exception. NetworkError is something is checked and retries in a few places so
that seems like a natural candidate.

Reviewed By: johansglock

Differential Revision: D26855444

fbshipit-source-id: f15c723293a416b5f44a6592927e3500f3b0b7d5
2021-03-08 08:06:00 -08:00
Stefan Filip
51367f0312 error: generalize NetworkError to all network read errors
Summary: Timeouts are another class of errors that are relevant.

Reviewed By: johansglock

Differential Revision: D26855442

fbshipit-source-id: 8ebb83714fa3d7a2f4efcbed8bd512c98301b49d
2021-03-08 08:06:00 -08:00
Andrey Chursin
a43f073d9d checkout: use async_vfs in native checkout
Summary: The goal is to reduce load on tokio scheduler by using threads & channels instead of spawning new task every time

Reviewed By: DurhamG

Differential Revision: D26801249

fbshipit-source-id: a8d9accc721c7ffc981fd538c06ab8cbd908f715
2021-03-05 21:47:51 -08:00
Andrey Chursin
0be8e8ce29 vfs: introduce AsyncVfs
Summary:
AsyncVfs provides async vfs interface.
It will be used in the native checkout instead of current use case that spawns blocking tokio tasks for VFS action

Reviewed By: quark-zju

Differential Revision: D26801250

fbshipit-source-id: bb26c4fc8acac82f4b55bb3f2f3964a6d0b64014
2021-03-05 21:47:51 -08:00
Andrey Chursin
6c7b0134a3 vfs: make clear_conflicts private
Summary: This function is now internal details of vfs, and is not used outside of it. Making it private so it won't be accessed outside

Reviewed By: quark-zju

Differential Revision: D26801251

fbshipit-source-id: 03434e235978fa0745128d90ea0c5975ea662ff1
2021-03-05 21:47:50 -08:00
Lukas Piatkowski
ad106958f2 eden/scm/lib: autogenerate all Cargo.toml files with autocargo
Summary: This diff removes the split between manually managed and autocargo managed Cargo.toml files in `eden/scm/lib`, now all files are autogenerated.

Reviewed By: quark-zju

Differential Revision: D26830884

fbshipit-source-id: 3a5d8409a61347c7650cc7d8192fa426c03733dc
2021-03-05 04:29:49 -08:00
Mateusz Kwapich
34b4972bdd skip deletion of root directory
Summary:
The GitTreeDict throws when we're trying to remove the root directory but there
are valid situations where the Tree becomes briefly empty. One of such is a
commit where all the files in the repo are moved. We process the deletions
first. Then we proceed with additions.

Reviewed By: DurhamG

Differential Revision: D26778717

fbshipit-source-id: 6caa1709ff2c5e78d4745336b0527af4bb20ec60
2021-03-05 02:56:29 -08:00
Durham Goode
ee45deeeb2 clone: add option to enable nativecheckout during clone
Summary:
Native checkout avoids the initial large prefetch, and instead does
small batch prefetches and immediately writes them to disk. This will prevent
the issue where data in the cache is evicted before it's used and has to be
redownloaded in a non-batch fashion later.

In a future diff, using native checkout will also allow us to serialize and
deserialize it's CheckoutPlan, so we can make clones resumable.

Reviewed By: quark-zju

Differential Revision: D26825773

fbshipit-source-id: 6fde90cd7578f6af5fdd4ac78004f7f63af4f287
2021-03-04 17:22:51 -08:00
Liubov Dmitrieva
d1c309ab17 remotebookmarks: check the remote when checking essential selective pull remote bookmarks
Summary: the list of configured default selective pull bookmarks are actually related to the hoist namespace I think when we consider what is protected and what is not.

Reviewed By: quark-zju

Differential Revision: D26814708

fbshipit-source-id: 566f363f1e6b88dd2ebc6e33cfb6eba596493988
2021-03-04 15:23:16 -08:00
Jun Wu
fd1bbe92f8 fsync: sync parent directories on POSIX systems
Summary:
On POSIX systems it's a good practice to fsync directories to ensure metadata
about a file is properly written.

Reviewed By: DurhamG

Differential Revision: D26822211

fbshipit-source-id: fca10c702b480f22020ad184eb55c8879999f0ee
2021-03-04 12:23:48 -08:00
Jun Wu
f10466c9f1 dirsync: do not wrap remote repo
Summary:
Typical mistake of `reposetup` - the `repo` object might be remote and lack of
local repo methods.

Reviewed By: DurhamG

Differential Revision: D26822960

fbshipit-source-id: 54fb95c51b09ef5021b0a10a93b4509e7aa4a115
2021-03-04 11:38:28 -08:00
generatedunixname89002005307016
0124627d4b suppress errors in fbcode/eden - batch 1
Differential Revision: D26753306

fbshipit-source-id: 335f45b534c6544eaf06e9d42fb16d758f8b0a5e
2021-03-04 11:03:06 -08:00
Stefan Filip
07750d0a3a notes: add 2019-10 Manifests, Past and Future
Summary:
I moved the source of the sphinx project to newdoc for newdoc/dev.
I updated the sphinx config for markdown to something that works for recent
versions.
PNG images rendered better than SVG for me.
I moved the TARGETS to newdoc.

Reviewed By: quark-zju

Differential Revision: D26801426

fbshipit-source-id: 3ae51e886d27f848f0f6a48c96056da607d8da45
2021-03-04 08:52:27 -08:00
Stefan Filip
1162e2ba0e gitignore: track Makefile under scm
Summary:
Mercurial uses Makefile here to build stuff. We want to track Makefile under
scm.

Reviewed By: quark-zju

Differential Revision: D26802165

fbshipit-source-id: 1fe8db13d50c07a6a0681180959eba22eaf8d486
2021-03-04 08:52:27 -08:00
Thomas Orozco
2a803fc10d third-party/rust: update futures
Summary:
Those newer versions of Futures have compatibility improvements with Tokio,
notably:

- https://github.com/rust-lang/futures-rs/pull/2333
- https://github.com/rust-lang/futures-rs/pull/2358

Reviewed By: farnz

Differential Revision: D26778794

fbshipit-source-id: 5a9dc002083e5edfa5c614d8d2242e586a93fcf6
2021-03-04 06:42:55 -08:00
Jun Wu
e70d89dd39 py3: fix absorb -i
Summary:
The previous fix D23357655 (d60e80796a) actually only fixes py2 absorb -i. On Python 3,
`b"-"[0]` is `45`, not `b"-"` like Python 2. Fix it again using `b"-"[0:1]`.

Reviewed By: singhsrb

Differential Revision: D26805315

fbshipit-source-id: 07ca850373a6bc49b561466ead478024631ce051
2021-03-03 20:50:31 -08:00
Xavier Deguillard
d9207cbe4e backingstore: do not initialize memcache for debug builds
Summary:
Memcache is dogslow to initialize, taking >30s on debug build. As a
consequence, this slows down every single test by that amount time, with the
guarantee that no blobs will be found in memcache, ie: a total waste of time.

On release builds, Memcache is significantly faster to initialize, so let's
only disable initializing Memcache for debug builds only.

Reviewed By: fanzeyi

Differential Revision: D26800265

fbshipit-source-id: 8b393c603414de68268fdadb385de177e214a328
2021-03-03 20:14:46 -08:00
Durham Goode
f22e274e94 py3: add py3 windows package to hgbuild
Summary:
Adds a new hg py3 windows package to hgbuild for publishing. Currently
the tests don't run. I'll do that next.

Reviewed By: quark-zju

Differential Revision: D26768336

fbshipit-source-id: bd4533abbbc1e1c861aa9995b7a3868a7f6a1a22
2021-03-03 19:52:54 -08:00
Jun Wu
4afd737525 hide: cleanup non-essential remote bookmarks automatically
Summary: Trigger the cleanup logic automatically if there are too many remote bookmarks.

Reviewed By: sfilipco

Differential Revision: D26802251

fbshipit-source-id: 1ab40c7be98c507865ea17001dc9775b6edf4446
2021-03-03 18:30:08 -08:00
Jun Wu
01035df3a4 hide: make --cleanup remove non-essential remote bookmarks
Summary: This is handy to make the `sl` output cleaner.

Reviewed By: sfilipco

Differential Revision: D26802250

fbshipit-source-id: 1b74f3d9a42ab6b0780f07bec59b03a5cd0ea6a9
2021-03-03 18:30:08 -08:00
Jun Wu
e33255b67d remotenames: add a command to clean up remote bookmarks
Summary: Add API and command to remove non-essential remote bookmarks

Reviewed By: sfilipco

Differential Revision: D26802252

fbshipit-source-id: bf715905f146f31aac19dccb90022bea31392323
2021-03-03 18:30:07 -08:00
Jun Wu
4abfeb66e8 remotenames: raise if key bookmarks point to unknown commits
Summary:
Previously remotenames pointing to unknown commits are just ignored.
If key remotename like master is ignored, it can cause very slow operations
in pull, etc. Let's just raise an exception in this case.

Reviewed By: DurhamG

Differential Revision: D26800236

fbshipit-source-id: 13be4af5499da1b9098b4ff1a6ef41c54092824a
2021-03-03 18:20:05 -08:00
Jun Wu
7a759b6075 visibility: remove public heads
Summary:
Remove public heads when using Rust changelog backends. This should address
some issues seen in commit cloud sync.

This is done at the metalog commit time so we get the latest "remotenames" data for
accurate "public()" set calculation.

Reviewed By: singhsrb

Differential Revision: D26792731

fbshipit-source-id: 00b894fee9804740d664dad0ac47be564820da33
2021-03-03 18:06:08 -08:00
Jun Wu
8142c5e5f1 windows: set output encoding to UTF-8
Summary:
The output encoding is used to render the graph log edges. With D26612487 (62ba7447f6), we
switched to Rust IO. The Rust IO requires UTF-8 data. So let's set
outputencoding to UTF-8.

Reviewed By: sfilipco

Differential Revision: D26799551

fbshipit-source-id: aa3e6420067d7c75bef47448e12e48f4cef56a84
2021-03-03 16:45:16 -08:00