Commit Graph

46709 Commits

Author SHA1 Message Date
Jun Wu
6a304509f0 Back out "[hg] configparser: inline the pest grammar"
Summary:
The pest codegen has some non-determinism (HashMap) that breaks buck build.

According to jsgf:

  This basically indicates some kind of non-determinism in the build. They're a pain, so I'd been hoping that we'd got them all.

  Yeah, pest is generating non-deterministic output, which will screw things up badly. The problem is:

  https://github.com/pest-parser/pest/blob/master/generator/src/generator.rs#L92-L93

    fn generate_builtin_rules() -> HashMap<&'static str, TokenStream> {
        let mut builtins = HashMap::new();

  is putting builtins into a `HashMap`, then:

  https://github.com/pest-parser/pest/blob/master/generator/src/generator.rs#L46

    rules.extend(defaults.into_iter().map(|name| builtins[name].clone()));

  emitting them in hashmap order. It needs to use a `BTreeMap` to make sure they're in a consistent order.

  (I didn't check whether there are other instances of this.)

Reviewed By: jsgf

Differential Revision: D17063573

fbshipit-source-id: c03adc3c6d50bd09ffbd44ca8dc7bc51d6cad28d
2019-08-26 18:31:44 -07:00
Jun Wu
3e4443737f lib: remove cargo workspace
Summary:
The workspace was added by D8741175 mainly to make build artifacts share a same
"target" directory.  D14606468 made `setup.py` write a `.cargo/config` that
specifies a sharable "target" directory. Remove the workspace since the crates
already share a "target" directory.

This makes it a bit easier to add new crates.

Reviewed By: singhsrb

Differential Revision: D17053934

fbshipit-source-id: d34781c796356b725ddce3453c1951a4d4133807
2019-08-26 17:34:34 -07:00
Jun Wu
05a2f169c8 bindings: disable LTO for faster builds
Summary:
LTO does not buy us much - libbindings.so is 13MB disabled; 11MB enabled.
But LTO increases build time significantly:

- 24 seconds lto=false
- 36 seconds lto=thin

Therefore just disable LTO for faster builds.

Reviewed By: xavierd

Differential Revision: D17053932

fbshipit-source-id: efb437dbd2a26756bcc3f785901d06b286724835
2019-08-26 12:30:37 -07:00
Jun Wu
c89cfad845 rust: set linker on linux to ld.gold
Summary:
`ld.gold` is much faster than `ld.bfd`.  That reduces `bindings` mtime-only change
debug build time from 10 seconds to 5 seconds.

Reviewed By: xavierd

Differential Revision: D17053933

fbshipit-source-id: 6d55a09723dd4d78d8a9db8c0721a18eabb6d435
2019-08-26 12:30:36 -07:00
Xavier Deguillard
2d396615a9 remotefilelog: remove duplicated reportpackmetrics
Summary: This is already called in makeunionstores, no need to call it a second time.

Reviewed By: quark-zju

Differential Revision: D16794078

fbshipit-source-id: ff4c51d64748dfe4160a26767af11f7ebeb82618
2019-08-26 11:38:54 -07:00
Stefan Filip
4be6f11ab5 localrepo: reorder remove operations so they happen before add
Summary:
The Rust Manifests prevent having both a file and a directory with the same
name. This is a problem for the localrepo commit logic which will first add
the new entry, be that file or directory, and then remove the old one.

Reviewed By: quark-zju

Differential Revision: D16775350

fbshipit-source-id: 4515e42f9d2b1b2bc4861c16a39d291225cddc2a
2019-08-26 10:48:08 -07:00
Stefan Filip
8a402cc939 manifest: update insert validation
Summary:
The insert code would be unclear in what kind of issue it ran into when
inserting files. Sometimes the file we want to insert is a directory and
other times it want to traverse a directory. This change makes those
situations clear along with some other corner case behaviors.

Reviewed By: quark-zju

Differential Revision: D16775354

fbshipit-source-id: 50ab6bc52b70cc5cef013d11050eb3cdf5b160a5
2019-08-26 10:48:08 -07:00
Stefan Filip
d4614d568d manifest: update manifest.remove to return None for dirs
Summary:
Updating the manifest implementation for remove with the intended API.
When I originally implemented remove I wasn't confident what was the
best way to implement remove. As I've gained more experience, I feel
confident that doing two iteration over the tree is a good approach
for this method. The first iteration should validate that the file
exists then the second iteration will actually traverse down updating
the nodes to mutable ephemerals.

Reviewed By: quark-zju

Differential Revision: D16775353

fbshipit-source-id: 8ebee9ca347efcb694a6d27c1eeae2c149643766
2019-08-26 10:48:08 -07:00
Stefan Filip
2e3860a3d9 manifest: fix get_link to return None for children of files
Summary:
`get_link` started as test function that broaden in scope but did not have
it's behavior updated as it started to be used more broadly.
No reason to error out when we request a path that has parent files in the
manifest.

Reviewed By: quark-zju

Differential Revision: D16775356

fbshipit-source-id: a320926100378f16d723ca204746906e79c7752e
2019-08-26 10:48:07 -07:00
Stefan Filip
98b32be924 manifest: remove the dirs function from the manifest
Summary:
There is no custom implementation for manfiest.dirs(). Generally speaking
the custom implementation is a good thing but doing the migration in the
current python codebase doesn't seem worth while at the moment.

Reviewed By: quark-zju

Differential Revision: D16775351

fbshipit-source-id: c428860d21088a50a0f754dc20d6ee224d2eae32
2019-08-26 10:48:07 -07:00
Stefan Filip
c36d604b3f bindings: add manifest.matches
Summary:
The C++ constructs a manifestdict for matches. The assumption is that the
method is expected to return a manfiestdict and other code breaks if that
is broken. We will have to go and fix this later.

Reviewed By: quark-zju

Differential Revision: D16775352

fbshipit-source-id: 300945ccdd6448fc6ffae16a954aac93d6f526be
2019-08-26 10:48:07 -07:00
Stefan Filip
dd8084d39b bindings: fix common iteration problems on manifest
Summary: Minor fixes.

Reviewed By: quark-zju

Differential Revision: D16775355

fbshipit-source-id: fc4bf436555d14c12c685f8779c6b700fb234a24
2019-08-26 10:48:06 -07:00
Durham Goode
3d5cc68c87 treemanifest: prevent fetches that pass the target as the base
Summary:
We've seen problems where we send requests to the server for a manifest
node and claim that we have that same node as the base node. This results in the
server sending no data back. This is obviously a bug in the client, since if
we're trying to fetch the node it means we don't have it and shouldn't use it as
a base. Our current best guess is that the node is in the index but the actual
data is corrupt, so getmissing() claims it exists, but reads actually fail.

Until we figure out the root cause, let's prevent findrecenttree from returning
the node as a base to itself.

Reviewed By: sfilipco

Differential Revision: D16996425

fbshipit-source-id: 04b0cfbe1c432749f2aeb63f98c7b59ab50facc0
2019-08-23 15:30:11 -07:00
Jun Wu
42d2ec27d7 configparser: inline the pest grammar
Summary:
The latest pest_derive supports inline grammar. Replace the ad-hoc codegen with
it.

Reviewed By: markbt

Differential Revision: D16866455

fbshipit-source-id: e9cae33d4d97eeb09afa212a6c0ae777f4bb193b
2019-08-23 11:09:39 -07:00
Mark Thomas
756ee47bf3 commitcloud: add blackbox logging of cloud changes
Summary:
Add blackbox logging of syncing to or from the commit cloud workspace,
including which changes are being submitted.

Also log when obsmarker fixup happens in blackbox.

Reviewed By: quark-zju

Differential Revision: D16961281

fbshipit-source-id: 0d0f675d77ab3446198703b31eea940dae3bdd85
2019-08-23 05:03:38 -07:00
Mark Thomas
804303efc3 visibility: add logging for when tracking is in use
Summary:
Add a debug log for when a repository is tracking visibility through visible
heads.  It's not enough to just check if `visibility.enabled` is true, as the
repo needs to be migrated to visibility tracking, which might not have happened
yet.

Reviewed By: quark-zju

Differential Revision: D16961282

fbshipit-source-id: 8dec04dcc07e72e09f61399f9be04bdee339e262
2019-08-23 05:03:38 -07:00
Jun Wu
b49ce531cc clidispatch: fix a minor case of debug command detection
Summary: This makes `config|debugconfig` as a debug command. Previously it was not.

Reviewed By: singhsrb

Differential Revision: D16866460

fbshipit-source-id: f1fcf0777d2fe6c1426f557ae1999710dd722109
2019-08-22 18:30:29 -07:00
Stefan Filip
62e7989ad9 treemanifest: add logging for fetches with basemfnode==mfnode
Summary:
It is unclear where we generate basemfnodes that get to be equal to mfnodes
in our requests to the server. This logging allows us to track the codepath
that is problematic.

Reviewed By: quark-zju

Differential Revision: D16969888

fbshipit-source-id: b2d0aa7b573180664ad5cf8d6909451f089ac4a5
2019-08-22 14:20:02 -07:00
Jun Wu
4a8387c725 hgext: use commit hashes to call other commands
Summary:
Search for `commands.(update|rebase)` and change them accordingly to avoid
triggering revnum deprecation messages.

Reviewed By: singhsrb

Differential Revision: D16964237

fbshipit-source-id: c12077c6e143766cba5264cc1fbe0d8cc15c69a2
2019-08-22 13:00:14 -07:00
Jun Wu
06859de4e2 prune: pass commit hash to commands.update
Summary: This removes a false positive detection of "revision number" usage.

Reviewed By: singhsrb

Differential Revision: D16963944

fbshipit-source-id: 2fce56bc2c9deefaceec3ec89d6e3c784ca9da48
2019-08-22 13:00:14 -07:00
Jun Wu
b87bd484e6 histedit: use commit hash internally
Summary: This removes a false positive detection of "revision number" usage.

Reviewed By: singhsrb

Differential Revision: D16955293

fbshipit-source-id: eb5196c694b105a10f9a4424ca87aed8bac46281
2019-08-22 13:00:14 -07:00
Jun Wu
3e8db21b78 restack: use commit hash internally
Summary: This removes a false positive detection of "revision number" usage.

Reviewed By: singhsrb

Differential Revision: D16955289

fbshipit-source-id: 799b6e548ad08e9b9fdcc5d2c7ca792c9eb5684b
2019-08-22 13:00:13 -07:00
Jun Wu
0668c348e3 amend/movement: disable revnum warning for internal use of rebase
Summary: This removes a false positive detection of "revision number" usage.

Reviewed By: singhsrb

Differential Revision: D16955291

fbshipit-source-id: b9d37dc4c18c438dc64e25f5aa4e8e72155da0bd
2019-08-22 13:00:13 -07:00
Jun Wu
a6ad29eadf phase: use commit hash internally
Summary: This removes a false positive detection of "revision number" usage.

Reviewed By: singhsrb

Differential Revision: D16955290

fbshipit-source-id: 5d8762c21b0ceb9b7576f220e0d177db3918abc1
2019-08-22 13:00:13 -07:00
Jun Wu
272e2948fa split: pass commit hash to the revert command
Summary: This removes a false positive detection of "revision number" usage.

Reviewed By: singhsrb

Differential Revision: D16955292

fbshipit-source-id: e32e242416474185700db5331610dd2faa78b7c4
2019-08-22 13:00:12 -07:00
Jun Wu
38bf97ab62 directaccess: avoid false positive detection of revnum usage
Summary:
Similar to the previous diff, opt out the revnum detection for the `repo.revs`
call used internally by directaccess to calculate revs.

Change smartlog to disable the revnum detection for its internal calculation.
Carefully choose the scope so `smartlog(1+2)` will still be warned.

Reviewed By: singhsrb

Differential Revision: D16954996

fbshipit-source-id: 2612055a8a9bb2fc9bdd1946c4bc565f967b22ca
2019-08-22 13:00:12 -07:00
Jun Wu
17f66b3e1f smartlog: avoid false positive detection of revnum usage
Summary:
If `smartlog()` is the user input, the revnum detection is enabled for the
whole scope of smartlog revset implemenation. The use of rev numbers triggers
the warning. But the user didn't use any rev number.

Change smartlog to disable the revnum detection for its internal calculation.
Carefully choose the scope so `smartlog(1+2)` will still be warned.

Reviewed By: singhsrb

Differential Revision: D16954997

fbshipit-source-id: 1a3d32c1c2bcba08bfac908623b6416cf1cd63a8
2019-08-22 13:00:12 -07:00
Aleksei Kulikov
88ba91d3dc snapshot: add negative tests of broken metadata
Reviewed By: markbt

Differential Revision: D16961999

fbshipit-source-id: 730766962fab1ed94bfa8991256ac963a78cf4c0
2019-08-22 09:32:04 -07:00
Aleksei Kulikov
b8088a8a9e snapshot: rename commands to non-debug ones
Summary:
Instead of `debugsnapshot` and `debugcheckoutsnapshot` we will use
`snapshot create|checkout`.

Reviewed By: markbt

Differential Revision: D16961037

fbshipit-source-id: 142d072672376c2ce6a248a4dceffdd94d56a4df
2019-08-22 09:32:04 -07:00
Aleksei Kulikov
216bd69c29 snapshot: add version info to snapshot metadata
Reviewed By: markbt

Differential Revision: D16941087

fbshipit-source-id: 10d1857060dafcda2005a14d731758a2221ad7fd
2019-08-22 09:32:04 -07:00
Jun Wu
adbe2b2b32 alias: ignore alias definitions with ":" in name
Summary:
Aliases with `:doc` in name are not real commands. Do not treat them as
commands.

The upstream patch https://phab.mercurial-scm.org/D5087 added other metadata
including `:help` and `:category`. We might end up having some in the future
so I blacklisted names with `:` in them, not just `:doc`.

Reviewed By: chadaustin

Differential Revision: D16955596

fbshipit-source-id: b6f3e1129b632e0ba43c800e7e6fdd2b52d3b40c
2019-08-21 21:13:38 -07:00
Jun Wu
8de5266806 hgmain: fix build with buildinfo enabled
Summary:
The type of `args` has changed. Tests did not capture the issue, since the code
block was not compiled using the default setup.

Reviewed By: sfilipco

Differential Revision: D16953736

fbshipit-source-id: a678f1354511f2d2ceb5faa7a4de0c0c6b03e84d
2019-08-21 18:02:40 -07:00
Jun Wu
7eb73196c4 clidispatch: avoid fs::canonicalize
Summary:
`fs::canonicalize` has unwanted side effect - resolving symlinks.
Use `util::path::absolute` instead. This resolves a side effect where

Note on Linux, `chdir` resolves symlinks and `getcwd` (aka.
`std::env::current_dir`) will return the resolved path.  On Windows
`std::env::current_dir` does not return the resolved path, at least
for the mapped drive created via the `subst` command.

Differential Revision: D16952484

fbshipit-source-id: 6969a1844020ff6b82de46416d8950ec40394159
2019-08-21 17:59:50 -07:00
Jun Wu
3e73af0a9a util: add a function to get absolute path without resolving symlinks
Summary:
On Windows, some people use the "map drive" feature to map a long path (ex.
`C:\long\path\to\repo`) to a short path (ex. `Z:\`) so their tooling can
handle some long paths.

In that case, resolving symlinks by `hg root` is undesirable.

Unfortunately, the Rust stdlib does not have a Python `os.path.abspath`
equivalent. There were some attempts (ex. https://github.com/rust-lang/rust/pull/47363)
but the corner cases (ex. symlinks) have made the problem much more
complicated.

There are some 3rd-party crates. But they are not a good fit:
- https://github.com/danreeves/path-clean/ (last commit fb84930) follows the golang plan9 idea. It does not have proper support for Windows paths.
- https://github.com/vitiral/path_abs/ (latest commit 8370838) reinvents many path-related types, which is an overkill for this usecase.

This diff implements the feature "reasonably" for both Windows and Linux, with
nasty corner cases (symlink) ignored.

Differential Revision: D16952485

fbshipit-source-id: ba91f4975c2e018362e2530119765a380f103e19
2019-08-21 17:59:50 -07:00
Jun Wu
1387b3f3bf eden: remove 'prefetch' method from HgServer
Summary:
It does not seem to get used anywhere. Its usage of `scmutil.revrange` can
potentailly trigger a false positive "legacy revision number used" warning.

Reviewed By: chadaustin

Differential Revision: D16910635

fbshipit-source-id: 0b84d4b6a85678c581ccf1754bfbefbc97d8c055
2019-08-21 16:39:07 -07:00
Jun Wu
b85df9caf2 clidispatch: move logic to read system and user config to configparser
Summary: The configparser crate is a better place for logic to load system and user config.

Reviewed By: sfilipco

Differential Revision: D16796396

fbshipit-source-id: c1ae5f85aa9839d89f5026279d699135dc1b442b
2019-08-21 12:59:29 -07:00
Jun Wu
8f349dbb17 clidispatch: make repo.shared_path non-lazy
Summary:
Move `repo.shared_path` handling to repo initialization time and store it in
the repo structure.

This makes `repo.shared_path()` cheap if it gets used frequently.

Reviewed By: sfilipco

Differential Revision: D16796401

fbshipit-source-id: e19f3381cc87b55500ea1d27fd918ccb16a71972
2019-08-21 12:59:28 -07:00
Jun Wu
08a80a4bcc clidispatch: move repo specific logic to repo.rs
Summary:
Some functions in `dispatch.rs` are about the "repo". Move them to a better
place - `repo.rs`.

The repo and config logic is coupled. A new enum `OptionalRepo` was added, to
make code easier to write - `Some(repo)` means the repo exists, and `repo` owns
the config. `None(config)` means the repo is missing, but the config is still
available.

Reviewed By: sfilipco

Differential Revision: D16796403

fbshipit-source-id: 2f4017a52296b629e990f85437b2cfdd7263b9e6
2019-08-21 12:59:28 -07:00
Jun Wu
19b4a7c3de clidispatch: rename InferRepo to OptionalRepo
Summary:
"infer" means "try to get the repo path from command line arguments like a full path".
The enum variant is really "repo is optional". Rename to clarify.

Reviewed By: sfilipco

Differential Revision: D16796399

fbshipit-source-id: 505d2a406a83e0006200ece63d360b119548d2dd
2019-08-21 12:59:28 -07:00
Jun Wu
b6101a12e0 clidispatch: rework error handling
Summary:
Change error type in clidispatch from `DispatchError` to `failure::Error`.

Pros:
- `failure` will attach a backtrace for free. (otherwise, backtrace handling is
  manual)
- Wrapping other errors (ex. `io::Error`, `cliparser::Error`) is optional.
  (otherwise, wrapping other errors is mandatory, and needs to be careful to
  not lose information)

Cons:
- No longer able to enumerate *all* possible error types. (but can still
  downcast to specific errors)

This seems to be a good tradeoff especially because of the backtrace handling - I
ran into a few issues where the location where the error happened really helped
debugging.

Since we can no longer enumerate all possible error types, the enum was changed
to individual structs to make the code shorter (ex. the struct can be downcasted
directly, instead of down-casting to the enum, then matching its variant).

The `HighLevelError` handling was simplified and moved to `hgmain`.

The new code path falls back to Python less aggressively, therefore some behaviors
were tweaked (ex. `-R` takes a bundle path).

Reviewed By: sfilipco

Differential Revision: D16796400

fbshipit-source-id: 1b17eb8d62503644b118c6b799778182bf222538
2019-08-21 12:59:28 -07:00
Jun Wu
7cce00694c cliparser: use lowercase text in errors
Summary:
This is more conistent with Mercurial style. And make them usable directly in
Rust code.

Reviewed By: sfilipco

Differential Revision: D16796397

fbshipit-source-id: 9016ea2b09fdf96b2b54138f5c8405caf96390f7
2019-08-21 12:16:36 -07:00
Jun Wu
253b03e87c cliparser: sort possibilities in AmbiguousCommand error
Summary:
This makes the "content" of the error stable. It is used in a later diff where
AmbiguousCommand gets handled by Rust directly instead of falling back to
Python dispatch.py.

Reviewed By: sfilipco

Differential Revision: D16796404

fbshipit-source-id: c439db14ec83c76c4762d3c627bfce1ea44bccf4
2019-08-21 12:16:36 -07:00
Jun Wu
0f8f66f8b4 clidispatch: make CommandDefinition::flags lazy
Summary:
In case there are many CommandDefinitions, we don't need all their flag
definitions until we decided to execute one of the commands.

Reviewed By: sfilipco

Differential Revision: D16796398

fbshipit-source-id: af205f59efd77fd7ff9eb4655d1f9167e2c350da
2019-08-21 12:16:36 -07:00
Jun Wu
0bc2824c82 clidispatch: make use of HgGlobalOpts
Summary: Convert global flags to `HgGlobalOpts` struct to make code shorter.

Reviewed By: sfilipco

Differential Revision: D16796407

fbshipit-source-id: b9d4c3dbec68c81908d439da4c353249347ca74a
2019-08-21 12:16:36 -07:00
Jun Wu
1623399c15 clidispatch: remove tests
Summary:
The tests are

- Mostly testing about configurations.
- Mostly depends on private functions.

ConfigSet already has a good test coverage, the tests do not provide much
value (aside from testing config override ordering, which is also covered
by hg tests). I'm going to change / remove the private functions. Remove the
tests to make changes easier.

Reviewed By: sfilipco

Differential Revision: D16796402

fbshipit-source-id: 56a8d55a0d1b0438bd2fcde5d3379d76f51dcd9d
2019-08-21 12:16:35 -07:00
Jun Wu
c508ada8b2 clidispatch: use define_flags! to define global flags
Summary: This makes it possible to simplify the code reading global flags.

Reviewed By: sfilipco

Differential Revision: D16796405

fbshipit-source-id: eb604470d052ef84b748d1e60270cacd410fc5da
2019-08-21 12:16:35 -07:00
Jun Wu
210bf49f5e clidispatch: replace Dispatcher with CommandTable
Summary:
The `Dispatcher` provides lots of features but its internal state only contains
the command table. Replace it with `CommandTable` and make the methods free
functions.

This makes function dependencies more cleaner, for example things like "locating
a repo", "getting the args" etc. won't require a `Dispatcher`.

A side effect of this change is the non-utf8 command line arguments are no longer
supported. This is already broken since our hg wrapper (accidentally) enforced
utf-8 command line. Therefore related tests are removed.

Reviewed By: sfilipco

Differential Revision: D16796395

fbshipit-source-id: a793ce7b8befe7caf62405c582fc932eb3daa099
2019-08-21 12:16:35 -07:00
Jun Wu
6aeaa40b40 clidispatch: extract CommandTable to a separate struct
Summary:
The dispatch logic couples with too much stuff - repo, parsing, command,
config. The command / registration part can be cleanly seprated.

This diff moves the core logic of CommandTable to command.rs.

Reviewed By: sfilipco

Differential Revision: D16796406

fbshipit-source-id: 5a33d6b6452537f07895bd9944122a3b3149d925
2019-08-21 12:16:34 -07:00
Aleksei Kulikov
e8c5b4830f snapshot: rename manifest to metadata
Summary: Snapshot `metadata` won't be confused with regular commit manifest.

Reviewed By: markbt

Differential Revision: D16929553

fbshipit-source-id: 466b167aa1cc30f137e0c0b4601d70876509e03c
2019-08-21 10:36:34 -07:00
Aleksei Kulikov
71a0402c8a snapshot: fix dirstate behavior during snapshot checkout
Summary:
At first, that aim was to create a good and consistent test plan of snapshots:
```
# Snapshot test plan:
# 1) Empty snapshot (no changes);
# 2) Snapshot with an empty manifest (changes only in tracked files);
# 3) Snapshot with a manifest (merge state + mixed changes);
# 4) Same as 3 but test the --clean flag on creation;
# 5) Same as 3 but test the --force flag on restore.
```
Then I discovered some bugs while writing the tests.
1) In case of an empty manifest the `snapshotmanifestid` extra should be an empty string, not `None`.
This is because of the extra serialization rules.
2) The initial approach for doing the checkout did some unnecessary operations with dirstate,
which required it to rollback later.
Now the overhead is gone, and the strategy of checking out is as follows:
```
1. Regular hg update to the (1st) parent of the snapshot;
2. Apply all the changes via mergemod.update without pushing it to the dirstate;
3. Register these changes in the dirstate;
4. Tie the second snapshot parent to the working context, if it exists.

Reviewed By: markbt

Differential Revision: D16929417

fbshipit-source-id: 0319ac0e81f80956cdef5b2a696b642cc782f48d
2019-08-21 10:36:34 -07:00