Commit Graph

159 Commits

Author SHA1 Message Date
Mark Thomas
105e0dccf6 obsolete: don't create autorels for the final successor
Summary:
Autorels attempts to detect the scenario where `P -> Q` and `X -> Y` are
being added, and there already exists a `P -> ... -> X` relationship.

In this case it will create a `Q -> Y` "copy" marker to express the fact
that `Q` should be copied.

However, this also triggers in the case where `Q == Y`, creating a revive
marker for `Q`.

Normally this is benign, as Q is probably visible anyway, however when there
are two commits associated with a diff that has been landed, pullcreatemarkers
can create two markers: `P -> L` and `X -> L`.  Since P and X are for the same
diff, there probably exists a `P -> ... -> X` relationship, and so autorels
attempts to make an `L -> L` marker.  This fails because L is public.

Differential Revision: D14891063

fbshipit-source-id: 3f076a003508dd7b7d17e3eb7cdaeb8ac09e6b15
2019-04-11 07:49:21 -07:00
Mark Thomas
9c9b28498c registrar: document 'subonly' option to the command decorator
Summary:
The `subonly` option in the command decorator indicates that a command must be
called with a subcommand.  Document this in the decorator documentation.

Differential Revision: D14876782

fbshipit-source-id: d121f94437e47cad2a7f3abe0faac300792c1c76
2019-04-11 07:49:21 -07:00
Mark Thomas
511b52863e visibility: support cloud sync
Summary:
Add support for explicit visibility tracking in commit cloud sync.

This means commit cloud reads the visibleheads and syncs these with the commit
cloud heads directly, removing the source of problems where obsmarkers disagree
on different hosts.

Commit cloud requires that the ordering of heads is maintained to get stable
ordering of new commits.  Update the visibleheads tracking to maintain
ordering, rather than using sets.

Finally, the calculation of the replacement node was slightly off.  This was
revealed in the new test case that is being added, so it is also fixed.

Differential Revision: D14876266

fbshipit-source-id: fe5b6bffd196d3bd74e7582e29484969495eac8e
2019-04-11 07:49:20 -07:00
Mark Thomas
0b42418c85 visibility: improve documentation
Summary: Add doc comments describing what the visibility functions do.

Reviewed By: DurhamG

Differential Revision: D14871230

fbshipit-source-id: e0117ed1357a3a14b9f42034bf51a6cbb38ba67b
2019-04-11 02:45:16 -07:00
Mark Thomas
d562032896 mutation: improve performance of obsolete check
Summary:
The computation of whether a commit is obsolete or not can be improved.

We can cache which commits are known to not be obsolete.

We can also have a cache for each filter type so that we only need to compute
obsolete nodes that match the filter.

Finally, when we need to compute all obsolete commits, we can start by looking
for commits which are made obsolete by only their closest successors, and then
filling back obsolescence to the predecessors of these obsolete commits.

Reviewed By: DurhamG

Differential Revision: D14858655

fbshipit-source-id: 1d03e214ad878ecb6ae548f80373702e2a184146
2019-04-11 02:45:16 -07:00
Mark Thomas
d803ebb3c7 visibility: make enabling or disabling of tracking a command
Summary:
Rather than implicitly upgrading or downgrading a repo based on the
`visibility.tracking` config option, add a new `hg debugvisibility` command to
do this explicitly.

Reviewed By: DurhamG

Differential Revision: D14871231

fbshipit-source-id: 73f4648408b3eca9ac12bd77e54d2d37ee342069
2019-04-11 02:45:15 -07:00
Mark Thomas
2f0922256b mutation: add debugmutationfromobsolete to backfill mutation records
Summary:
Add `hg debugmutationfromobsolete` which backfills mutation information from
obsmarkers.

Reviewed By: DurhamG

Differential Revision: D14603199

fbshipit-source-id: 64458c52ba0e9f7bc9b38abeb94660cd1423fca7
2019-04-11 02:45:14 -07:00
Mark Thomas
87d3ef1330 mutation: record predecessors and operation in commit loginfo
Summary:
Record the commit predecessors and the mutation operation in the commit loginfo
so that it can be logged to telemetry.

Reviewed By: quark-zju

Differential Revision: D14798032

fbshipit-source-id: 9c4ac1a4df3c91087c776d1f8e5fca94713b0390
2019-04-10 15:16:17 -07:00
Stanislau Hlebik
0c019f20fd remove accidentally added lines
fbshipit-source-id: 70f4621d5adb859d39bf3bb8adb2ff84aa75ac5e
2019-04-10 00:45:56 -07:00
Stanislau Hlebik
dfdfbf232e experimental stash test
fbshipit-source-id: 5ff859300513dbb13889f3fc3bb781f7d56b85b6
2019-04-10 00:36:40 -07:00
Arun Kulshreshtha
de2605bf4d edenapi: make data and history batch sizes separately configurable
Summary: Make the batch size of data and history requests independently configurable, since data responses are typically much larger than history responses (since the former contains actual file data whereas the latter is only metadata).

Differential Revision: D14859686

fbshipit-source-id: c87c31f3e6611a55ae712e7f0ed9bb392d31a579
2019-04-09 17:00:01 -07:00
Arun Kulshreshtha
9fdd71e4df edenapi: use curl multi interface
Summary: Use the curl multi interface to fetch multiple batches of files or history entries concurrently.

Differential Revision: D14718547

fbshipit-source-id: c5a740c7e9106b719e825540f8182be31a72bae7
2019-04-09 14:59:44 -07:00
Arun Kulshreshtha
6705e2d120 bindings: allow choice between edenapi backends
Summary: Allow users to configure which HTTP client backend to use for the Eden API via the `edenapi.backend` config option. Valid options are `curl` and `hyper`, with `curl` being the default.

Reviewed By: quark-zju

Differential Revision: D14657871

fbshipit-source-id: 7a9972d2380fbbd5ed62d1accae764dc03ca4c29
2019-04-05 17:34:14 -07:00
Arun Kulshreshtha
6c8c87dea1 edenapi: add curl-based client
Summary:
Add a new Eden API client based on libcurl (via the rust-curl crate). This should help us work around issues with Hyper.

This implementation is based on curl's "easy" API, and is intentionally naive. I intend to update it to use curl's "multi" API to send several concurrent HTTP requests per operation in a later diff.

Differential Revision: D14656756

fbshipit-source-id: 1f71074506844104f0f3237023b38317a7f41979
2019-04-05 17:34:14 -07:00
Stefan Filip
9751716173 commitctx: refuse to commit files names that can't be interpreted as UTF-8
Summary: This is a stop gap measure while we move internal path encodings to UTF-8.

Reviewed By: quark-zju

Differential Revision: D14791761

fbshipit-source-id: e4d5a845aa4f28ed0744cf80021fd1446f3d6129
2019-04-05 16:30:00 -07:00
Stefan Filip
60e885a890 localrepo: import error module as errormod
Summary: `commitctx` takes error as a named parameter shadowing the module import.

Reviewed By: quark-zju

Differential Revision: D14805901

fbshipit-source-id: 359a6d19d6a08b3c65ebf1af8a5e5baaec6711a2
2019-04-05 16:30:00 -07:00
Marla Azriel
36a67da6f7 commands: help text for commit
Summary: Updated help text for hg commit

Reviewed By: kulshrax

Differential Revision: D14811336

fbshipit-source-id: dbc26b3409094b1f70aca01b0f9ff89693ce7861
2019-04-05 16:23:14 -07:00
Jun Wu
31c8ecbdf4 dispatch: print shorter crash header
Summary:
Change the crash header to just a single line without blaming extensions or
Python. This makes the crash log shorter, and easier to read when there are
crashes in run-tests.py diffs.

Remove `ui.supportcontact` since it's no longer used.

Reviewed By: sfilipco

Differential Revision: D14393982

fbshipit-source-id: 3ede8d3d3d8fd5d125944a9a750350d3ce356b14
2019-04-04 23:38:34 -07:00
Mark Thomas
9867fdb893 cmdutil: add checkout identifier to commit loginfo
Summary:
Add the checkout identifier to the loginfo for commit contexts that take data
from the working copy.

Reviewed By: quark-zju

Differential Revision: D14751601

fbshipit-source-id: 6ddcbf17b970ee24237edf501ae3d7fc8d320193
2019-04-04 11:29:50 -07:00
Mark Thomas
5274006a31 context: add loginfo for committable contexts
Summary:
Add `loginfo`, a new field to committable contexts, the contents of which will
be logged when the commit is committed to the repository.

Initially the logging only includes the commit node and author, but more things
will be added later.

Reviewed By: quark-zju

Differential Revision: D14751602

fbshipit-source-id: 2432dfb92dd2489cb8ad3464316a42ebaa79657c
2019-04-04 11:29:50 -07:00
Mark Thomas
fc9106e076 dirstate: add checkoutidentifier to identify and correlate checkouts
Summary:
When the user checks out a new commit (either by updating to it, by
creating a new commit, or by amending or rebasing the current commit), create a
unique identifier for that checkout.

Log this identifier at the start and end of command processing, and allow other
tools to also query and log the identifier.

This allows both Mercurial commands and other commands that log the identifier
to be correlated with a particular checkout.

Reviewed By: quark-zju

Differential Revision: D14648523

fbshipit-source-id: 2fad79c3010f5fad1a0e180e3d3d6d9c0a7f8e85
2019-04-04 11:29:49 -07:00
Mark Thomas
528a5b064e util: extract makerandomidentifier from clienttelemetry
Summary: Extract the `makerandomidentifier` function from the `clienttelemetry` extension so that we can use it elsewhere.

Reviewed By: quark-zju

Differential Revision: D14725866

fbshipit-source-id: 0b3ed2e8c9698e892ad0f524f6dde7bf7337f85c
2019-04-04 11:29:49 -07:00
Mark Thomas
9cf99b9293 sampling: log working directory parents before and after commands
Reviewed By: quark-zju

Differential Revision: D14648522

fbshipit-source-id: 7c67ae3fa0542dd38fd87c5e2a2b421f73e4e12d
2019-04-04 11:29:49 -07:00
Arun Kulshreshtha
d5c9b17b05 bindings: turn off non_camel_case_types warnings
Summary: We're intentionally exporting Python classes with names that match Mercurials convention of `allonewordlowercase`, so we should turn off this lint.

Differential Revision: D14760996

fbshipit-source-id: cf79e8ebb6a01bd7bcfb062754cb034d94ac493b
2019-04-03 18:27:28 -07:00
Marla Azriel
39e4eb08af commands: help text for status, add, remove
Summary: Updated help text for hg status, hg add, hg remove

Differential Revision: D14627242

fbshipit-source-id: 83367838273e861b9ca6e73e588b175219dbe187
2019-04-02 14:26:03 -07:00
Arun Kulshreshtha
1472ff1efa edenapi: refactor Builder into Config
Summary:
Rather than having a `Builder` struct that knows how to build just one kind of Eden API client, let's have a common `Config` type that can be potentially passed to the constructor of several different client implementations.

This will allow the same config code to be re-used across different client types, as seen later in this stack.

Differential Revision: D14656757

fbshipit-source-id: 883ffd2dc0302ebe08960f079c113e2d0da2d2ca
2019-04-01 20:15:38 -07:00
Jun Wu
74bade4e9a progress: reduce overhead when a nested bar is used in a loop
Summary:
Before this change, the following command will redraw the progress bar
frequently:

  ./hg debugprogress  --sleep 1 100000 --config progress.refresh=1 --nested

After this change, the above command only redraws once per second.

This should reduce overhead in `repack.py:repacker.repackdata`.

Differential Revision: D14693924

fbshipit-source-id: baa281ba2e262d2ef8e83c2460f7edf5400db1af
2019-03-29 20:32:22 -07:00
Jun Wu
9cba88f929 debugprocess: add a --sleep parameter
Summary: This makes it easier to debug real world progress related issues.

Differential Revision: D14691592

fbshipit-source-id: 2a653f657b6ba9f9369f1551845949ac7e4d0b9d
2019-03-29 20:32:22 -07:00
Durham Goode
b5d29d31b4 pushrebase: fix pushrebase for treeonly lfs pushes
Summary:
When doing a pushrebase, the server needs to mount a bundle over the
exist repository to run hooks. With lfs enabled this uses changegroupv3, and
with a treeonly client no manifests are sent as part of the changegroup (they
are instead sent as a separate part).

The combination of these three things hits an edge cases where if there are no
manifests in a changegroupv3 bundle, the bundlerepo class may not skip the
entire manifest part, and therefore the bundle appears to have no files in it,
even though the data is there.

This only happens if the server is in treeonly mode, or during pushrebase hooks
where we fake treeonly mode briefly. Since we don't have any server repos that
are treeonly and lfs yet, this didn't break us, but it would have. We caught it
because some of our hooks are affected when they try to read file contents.

The fix is to account for the cgv3 case in the bundle manifest consuming code.
Eventually we can get rid of all of this logic once we get rid of revlogs and
changegroups.

Differential Revision: D14676674

fbshipit-source-id: 86ac1ff4a92e37c4da3767a66760b55b658c836f
2019-03-29 13:57:54 -07:00
Xavier Deguillard
9d7bf6b735 dispatch: SIGPIPE do not exist on Windows
Summary:
On Windows, SIGPIPE isn't present in the signal module. This caused the exit
code to raise an exception when trying to install a handler for it.

Reviewed By: DurhamG, strager

Differential Revision: D14655142

fbshipit-source-id: 68ddfc5250a4b706c5bc6a7ad1add37e1ccbe26e
2019-03-27 19:13:55 -07:00
Mark Thomas
4b3fe9d147 mutation: compute fate only for obsolete commits
Summary:
When changes are undone so that an earlier view of the repo is present, fate
should only be calculated for commits that are obsolete in the current view.

This makes smartlog look correct after `undo` commands have been used.

Reviewed By: quark-zju

Differential Revision: D14603198

fbshipit-source-id: 3adaad94b9fe80e8a622d469513c784b772ee235
2019-03-27 04:49:14 -07:00
Mark Thomas
f008868a63 mutation: cache obsolete computations
Reviewed By: quark-zju

Differential Revision: D14603197

fbshipit-source-id: b7f9dc099f5a26663cbda970b58dd7d3b65159a8
2019-03-27 04:49:13 -07:00
Mark Thomas
ab9eb302f2 mutation: record mutation entries inside transactions
Summary:
Record mutation entries inside transactions.  This ensures that they only get
written when the transaction completes.

Indexedlog doesn't have support for pending writes, so mutation entries are not
visible to hook scripts.

Reviewed By: quark-zju

Differential Revision: D14566781

fbshipit-source-id: eb4c7bbd3878df82e8e7096a69509525f9fb93c0
2019-03-27 04:49:13 -07:00
Mark Thomas
29b30257b2 mutation: remove mutationcache
Summary:
Remove the last parts of the mutationcache, relying directly on the indexedlog
indexes.

The set of obsolete commits is now computed on the fly - we will cache this
later.

Reviewed By: quark-zju

Differential Revision: D14566780

fbshipit-source-id: 770d0d9d507bed982e88518dcf63a4c2d6b46a69
2019-03-27 04:49:13 -07:00
Mark Thomas
161db7bd2f visibility: clear filteredrevcache when visibility changes
Summary:
When visibility changes, the cache of filtered revs is no longer valid and must
be cleared.

Reviewed By: quark-zju

Differential Revision: D14566777

fbshipit-source-id: 681a73c126f3258baa7f61157aba557ea8aceeca
2019-03-27 04:49:13 -07:00
Mark Thomas
1a57ee38e2 mutation: move store to localrepo unfilteredproperty
Summary:
To prepare for transaction support, we need to move the store out of the cache
object and into its own property on the localrepo.

Reviewed By: quark-zju

Differential Revision: D14566776

fbshipit-source-id: 8fff03a86953fb60ed06dbbdcdd0bffc379bd047
2019-03-27 04:49:13 -07:00
Mark Thomas
6d84ff8825 mutation: create mutation store entries for local commits
Summary:
Computing mutation entries for all local commits is expensive when there are
lots of local draft commits.  Ideally we would have an indexed changelog that
would make these lookups fast, but until we have that, put entries in the
mutation store for these commits to take advantage of the fast lookup there.

Reviewed By: quark-zju

Differential Revision: D14566782

fbshipit-source-id: cc3a05715337a510a65d8ff436c59d16d0f0447e
2019-03-27 04:49:12 -07:00
Mark Thomas
4cef0e2f38 mutation: remove instabilities revsets
Summary:
The computations for instabilities revsets (`orphan`, `contentdivergent` and
`phasedivergent`) are complex, and aren't really necessary for the way we use
mutation information.

The `orphan` revset can be implemented as an alias for `obsolete():: -
obsolete()`.  The other conditions can be detected on a case-by-case basis
later if we need them.

Reviewed By: quark-zju

Differential Revision: D14566784

fbshipit-source-id: 60b8443ad4c0c82d8250d8e9a10e73fae770daa8
2019-03-27 04:49:12 -07:00
Mark Thomas
d6cf315f8f mutation: prefix predecessor hashes with hashing scheme
Summary:
To support future alternate hashing schemes, predecessor identities should be
prefixed with the hashing scheme in use.

Currently there is only one hashing scheme: `hg`, which is the Mercurial
hashing scheme.

Reviewed By: quark-zju

Differential Revision: D14566778

fbshipit-source-id: baaaf2f078886a1cc7ac20d12923a63b4da09db6
2019-03-27 04:49:12 -07:00
Jun Wu
d03f2d26c2 manifest: drop manifestv2 support
Summary:
The upstream has removed it in https://phab.mercurial-scm.org/D2393. Do the
same.

The deleted C++ code seems to leak `Py_False` if `usemanifestv2` is not set.

Reviewed By: singhsrb

Differential Revision: D14611525

fbshipit-source-id: d828526c31aaa861d100a846bba79d1f5898e245
2019-03-26 13:32:45 -07:00
Jun Wu
f3d5b0e27e revlog: drop v0 and v2 support
Summary:
A user hit cryptic error message:

  File "revset.py", line 423, in ancestor
    ancestors = repo.changelog.index.ancestors
  AttributeError: 'list' object has no attribute 'ancestors'

That was because they have run `rm -rf` and the `.hg` directory was mostly
removed. `hg` falls back to revlog v0, and the code in question obviously
dislikes revlog v0.

Change the error message to just say repo corruption.

Since there are code depending on revlog v1, and we don't use v0 or v2. Remove
both of them.

Reviewed By: singhsrb

Differential Revision: D14610769

fbshipit-source-id: 9015b74989d8fa4ad50bc36ec48bca26a1faa361
2019-03-26 13:32:45 -07:00
Durham Goode
73683c048e tracing: add various tracing across the code base
Summary:
Now that we have perftracing infra, let's trace a bunch of likely
problem spots.

Reviewed By: sfilipco

Differential Revision: D14426367

fbshipit-source-id: 354a241aa9ac5d75d34062a9838d581b4f46746f
2019-03-25 19:31:13 -07:00
Jun Wu
077313f1af dispatch: move deferred to atexit handler
Summary:
See D14606986 for context. `repo.close` should be protected by not crashing on
stdout or stderr write errors (SIGPIPE or EPIPE). Move `deferred` to `atexit`
handler to get the protection.

Test changes are caused by ordering changes.

Differential Revision: D14607407

fbshipit-source-id: 5a42aefcec395f48b8ecb67426429ef2e41f5666
2019-03-25 18:33:05 -07:00
Jun Wu
7873db8242 ui: silent stdio write errors during atexit
Summary:
This is the first diff of an alternative fix of D14528603. See also D14603974
for context.

We'd like atexit handlers to not crash with EPIPE or SIGPIPE when writing to
stderr or stdout. Therefore disable SIGPIPE signal handler and patch ui methods
to do so.

Differential Revision: D14607159

fbshipit-source-id: 274c5174813d402a7e0b8b5be7c8fcb0524fcdb3
2019-03-25 18:33:05 -07:00
Jun Wu
a3c4fd17e8 Back out "[hg] ui: do not enable SIGPIPE when running the pager"
Summary:
akushner and I both hit an issue where `hg log` hangs for a long time after
exiting the pager by pressing `q`. That was caused by D14528603. Revert it to
fix it.

The root cause is apparently a bug in Python 2 (or C stdlib?):

  # a.py
  import signal, os
  signal.signal(signal.SIGPIPE, signal.SIG_IGN)
  out = os.fdopen(1, 'wb', 1)
  while True:
      out.write(b"a\n")
  # shell
  python2 a.py | head -n 1
  # hangs

Python 3 does not have the issue.

Reviewed By: singhsrb

Differential Revision: D14603974

fbshipit-source-id: 3b4d716133853632e56dfeb59b1858f926a2c146
2019-03-25 18:33:05 -07:00
Durham Goode
ce423933c3 tracing: log perftraces to the blackbox
Summary:
Now that we're recording perftraces, let's log their results to the
blackbox if the command exceeds a configurable threshold.

Reviewed By: sfilipco

Differential Revision: D14426366

fbshipit-source-id: 24f261aaa2b089dbc959d709e51de1c0359d976d
2019-03-25 17:12:42 -07:00
Durham Goode
f2b414c324 tracing: add tracing for command dispatch
Summary:
Adds tracing for the command dispatch.  All other traces will be a
child of this root level trace.

Reviewed By: sfilipco

Differential Revision: D14426369

fbshipit-source-id: 2b996e027a2d6b163162bdceba571a3ee2186f6b
2019-03-25 17:12:42 -07:00
Durham Goode
31934386ed tracing: add ascii rendering for perftrace module
Summary:
The previous diff added functions for collecting data. Now let's add a
way to render it to ascii.  Future renderers may render this to json or the
Chrome trace format.

Example output:
```
 0.0  hg rebase -d 'master~100' -s 8342640303e2 (43.4s)
 0.0    Repo Setup (0.0s; local)
 0.0    Rebase (42.5s; disk-fallback)
 0.0      --missing-- (8.0s)
 8.0      Calculate Updates (34.1s)
 8.1        Prefetch Files (0.0s)
   :          * Keys: 0
   :          * Missing Data: 0
 8.1          LFS Prefetch (0.0s)
 8.1        --missing-- (33.2s)
41.3        Prefetch Files (0.0s)
   :          * Keys: 2
   :          * Missing Data: 0
41.3          LFS Prefetch (0.0s)
41.3        Check Unknown Files (0.0s)
42.1        Status (0.0s)
   :          * A/M/R Files: 2
42.1        Prefetch Files (0.0s)
   :          * Keys: 1
   :          * Missing Data: 0
42.1          LFS Prefetch (0.0s)
42.1      Prefetch Files (0.0s)
   :        * Keys: 0
   :        * Missing Data: 0
42.1        LFS Prefetch (0.0s)
42.1      Apply Updates (0.1s)
   :        * Actions: 2
42.2      Status (0.0s)
   :        * A/M/R Files: 2
42.2      Prefetch Files (0.0s)
   :        * Keys: 1
   :        * Missing Data: 0
42.2        LFS Prefetch (0.0s)
43.4
```

Reviewed By: sfilipco

Differential Revision: D14426370

fbshipit-source-id: 338348297372874b2d8f08dd07b411b90d33a5fa
2019-03-25 17:12:42 -07:00
Durham Goode
cdaf9a1a18 tracing: add initial perftrace module
Summary:
Adds the initial perftracing infra. It's just a collection of functions
for starting and ending tracing and associating data with the traces.

Reviewed By: quark-zju

Differential Revision: D14426368

fbshipit-source-id: 72a2fad901ff52877f3958db2b924d95f45eeac7
2019-03-25 17:12:42 -07:00
Xavier Deguillard
b7c4ad771b eden: properly close repo in debugedenimporthelper
Summary:
Closing a repo is required to commit the pending transactions, or to run a
background repack when packfiles were downloaded. Usually, the close method is
called in dispatch after a command is run and a valid repository was passed via
-R. The debugedenimporthelper however may open the repository manually, and
since it doesn't close it, we may end up with a large number of packfiles.

Let's manually close it to avoid this issue.

Reviewed By: simpkins

Differential Revision: D14531514

fbshipit-source-id: 9b2d1648775e4a2c9eea4c232bf0ae4fee43297d
2019-03-25 10:27:48 -07:00