Commit Graph

5633 Commits

Author SHA1 Message Date
Muir Manders
7bbc2e1517 lfs: only log hash mismatch if size matches expected
Summary: In production we do currently see some cases of lfs_read_hash_mismatch. I realized it may be due to missing chunks rather than bad chunk integrity. The new "skip_hash_on_read" path is still able to detect size mismatch, so tweak things to only log mismatch message if the sizes match.

Reviewed By: quark-zju

Differential Revision: D33928640

fbshipit-source-id: 5b63db2b05bc7b53779cdb5ae0041a7d22bb3962
2022-02-02 09:50:03 -08:00
Yan Soares Couto
2c677eb49c Default exchange.httpcommitlookup to True
Summary:
## Context
See D33771940 (d274a0f575)

## This diff
I move `exchange.httpcommitlookup` to True.

Reviewed By: quark-zju

Differential Revision: D33892116

fbshipit-source-id: 48a4b5b9effaeca0df62b56c173cb53ad231c497
2022-02-02 08:42:48 -08:00
Yan Soares Couto
4e25a7ef08 Default httpbookmarks to True
Summary:
## Context
See D33771940

## This diff
I move `pull.httpbookmarks` and `infinitepush.httpbookmarks` to True.

Reviewed By: quark-zju

Differential Revision: D33891956

fbshipit-source-id: cbc3d92aa7ac7ac9edf90b4f99ab9804e9d750be
2022-02-02 07:43:38 -08:00
Yan Soares Couto
d274a0f575 Default pull.httpmutations to true
Summary:
## Context

There are some config options that have been rolled out to 100% of users for some time, but they're still not used in tests by default, which makes our coverage worse.

The best way to change the behaviour of hg regarding config is to change the default value on `configitems.py`, and then (eventually, when this rolls out everywhere), delete them from the configerator overrides config.

The other option would be to leave them in configerator (or in eden/scm/lib/configparser/src/fb/mod.rs), but then they would also need to be replicated in all tests that want to use it, or at the very least duplicated in the base hgrc for integration tests, which is very prone to errors (or simply forgetting to do it, the state we are in today).

Notice that this still allows us to set them to false via configerator quickly, if necessary.

## This diff

This diff defaults `pull.httpmutations` to true: It was previously always set to true on configparser/src/fb/mod.rs

Reviewed By: quark-zju

Differential Revision: D33771940

fbshipit-source-id: 987db9cfaff720645eac4ad65edf560f875b3655
2022-02-02 07:43:38 -08:00
Jun Wu
472a1148b1 git: support bundle and unbundle
Summary: Delegate to git's own bundle implementation for bundle and unbundle.

Reviewed By: DurhamG

Differential Revision: D33895676

fbshipit-source-id: 3d17160c6f83c2f43ff41bcf3f74c00aecded630
2022-02-01 16:14:02 -08:00
Jun Wu
26d921b8c1 git: support older git by avoiding init -b
Summary:
The flag is not supported by git older than 2.28. There are still places
running older git. Change the command from `git init -b x` to
`git -c init.defaultBranch=x init` so it becomes compatibile with older
git.

We don't use the default branch. So set it to `_unused_branch` to
make it more explicit.

Reviewed By: DurhamG

Differential Revision: D33851129

fbshipit-source-id: 93534362b02353728d72d0ebef31c6a25737398f
2022-02-01 16:14:01 -08:00
Jun Wu
ed6b21bbce git: disable treemanifest prefetch
Summary:
With a local git store all trees are local. There is no need to prefetch.
Forcing prefetch trees would cause errors, since the protocol is incompatible.

Reviewed By: DurhamG

Differential Revision: D33837781

fbshipit-source-id: c41281ac96aff1501bd056901213b912a77f6c31
2022-02-01 16:14:01 -08:00
Jun Wu
d2e54864af git: update smartlog.repos config
Summary: It is a left-over from D33518822 (b897a19073).

Reviewed By: DurhamG

Differential Revision: D33837780

fbshipit-source-id: f42fb7f9099ab4cb388c7bd8f038839fd6976cbf
2022-02-01 16:14:01 -08:00
Jun Wu
0c883afe41 git: support clone --git
Summary:
Previously `clone` has to use git URLs with `git+` prefix in scheme such as
`git+ssh://...`, `git+file://...`, `git+https://...`. That was the same
behavior as the `hg-git` extension but is inconvenient for scp-like URLs
like `git@github.com:git/git.git` (provided by GitHub). That URL needs to be
translated to ssh form `git+ssh://git@github.com/git/git.git` manually,
which can be annoying.

Let's use a `--git` flag so `clone --git` can use the scp-like URL directly.
The `--git` flag also makes `clone` more consistent with `init`.

Reviewed By: DurhamG

Differential Revision: D33837782

fbshipit-source-id: 7fe11be6cf63932bde1f400cdfc2fc8120f5067a
2022-02-01 16:14:01 -08:00
Durham Goode
f88bf230ca rage: remove pack output from hg rage
Summary:
Packs are long dead. Let's remove the logging for hg rage. This logic
also doesn't work on Windows, due to depending on cmd commands, resulting in
some errors in the hg rage since we now use powershell.

Differential Revision: D33925868

fbshipit-source-id: c071f8f5685dbf7a9174cad31a0a8cd99a84cf88
2022-02-01 16:06:57 -08:00
Jun Wu
09d7d3785f eden: avoid slow linkrev scan paths
Summary: See the added comments and D33926639.

Differential Revision: D33927449

fbshipit-source-id: d60ff5410a6f878e3d0c85f7ad47072687e8e285
2022-02-01 15:37:47 -08:00
Muir Manders
17c76c1e4e nativecheckout: fix "-C" to always clear mergestate
Summary: There was an edge case where native checkout "up --clean" wasn't clearing the merge state if the working tree was clean. Add a check so we use fallback non-native code path if there is an active merge state.

Reviewed By: quark-zju

Differential Revision: D33893832

fbshipit-source-id: a0e4a8d23818a1e40582cede0afee62ac53cb680
2022-02-01 15:18:20 -08:00
Saul Gutierrez
a237ae47e7 repo: move the repo module to its own crate
Summary: Since `Repo` will soon contain more logic on its own not related to `clidispatcher`, we are moving its own crate.

Reviewed By: DurhamG

Differential Revision: D33883867

fbshipit-source-id: 94414e1e2692962bf491cc2abb4e1d3799cbcc03
2022-02-01 11:32:42 -08:00
Jan Mazur
c9e0a0bc1b remove x-x2pagentd-ws-over-h1=1 header
Summary: Due to a bug in the x2pagent with websocket connection polling in h2 we had to force h1. Now it's fixed and the new version of the agent with the fix should be rolled out to (almost - it's corp) 100% of users.

Reviewed By: quark-zju

Differential Revision: D33842574

fbshipit-source-id: aaf5c66143806a900253120a9f750e37ebe55b80
2022-02-01 03:13:47 -08:00
Muir Manders
dd78f808fb sigtrace: allow manually sigtracing long running commands
Summary: On Windows we apparently can't send a USR1 to trigger a trace on an eden helper process. This commit makes it possible to touch a file to trigger a particular long running process to write out its sigtrace.

Reviewed By: DurhamG

Differential Revision: D33904056

fbshipit-source-id: d42f6c071bcfaa8dfc352f65df31bdc5ea201823
2022-01-31 17:30:13 -08:00
Muir Manders
c7761fb8cd sparse: accept relative sparse profile names
Summary: Make "sparse enable" normalize profile names. It is not clear it wants a canonical path from the root of the repo, so now it makes some attempts to convert the given path(s) to that form. It now works with paths relative from cwd, and normalizes the path separator.

Reviewed By: quark-zju

Differential Revision: D33851998

fbshipit-source-id: 91a3d59b91266bb8941408d23d5021fa1f9e7025
2022-01-31 12:21:43 -08:00
Alex Hornby
2613ad5742 remove generated code for hgclientconf
Summary:
Point the thrift-types at the rust target, no need for the checked in generated code anymore

Saves generation, gives quicker signal on thrift changes (no codegen step), and stops thrift lib vs generated thrift code mismatches

Reviewed By: yancouto

Differential Revision: D33843588

fbshipit-source-id: 856faeb16dc0a2a42f2f07fd3005079a01b6e7bd
2022-01-31 10:34:32 -08:00
Andres Suarez
20675f6fc9 Update indexmap to 1.8.0
Summary: Non-breaking changes. https://github.com/bluss/indexmap/blob/1.8.0/RELEASES.rst

Reviewed By: Imxset21

Differential Revision: D33887730

fbshipit-source-id: 60e016f1f4429407b1f8232f3e5eefe5f89965fd
2022-01-31 07:48:41 -08:00
Muir Manders
06eb4d9977 pipe "fb" cargo feature down to clientinfo
Summary: I moved some stuff around and am having trouble getting everything to be happy. One thing I forgot was to actually pipe the "fb" feature down from the entry point crates (hgcommands and backingstore).

Reviewed By: xavierd

Differential Revision: D33852732

fbshipit-source-id: aabdd145452e66435e130fcbab0ba8a80f9d943a
2022-01-31 02:09:09 -08:00
Muir Manders
57d6b8787e nativecheckout: add total to "recording" progress bar
Summary: Replace "recording" spinner with a true progress bar.

Reviewed By: quark-zju

Differential Revision: D33796497

fbshipit-source-id: 2399b20af3c65ec91031b417413010d36d7a6101
2022-01-31 02:09:07 -08:00
Alex Hornby
b39d4ea2d1 updated expected config hash
Summary: Update the expected hash

Reviewed By: HarveyHunt

Differential Revision: D33843474

fbshipit-source-id: 4a14744e1d6ed15042b12f62f45634fa75591ffe
2022-01-31 02:09:07 -08:00
Jan Mazur
6d01fa3d6c make test-cats.t and test-advice-headers.t not flaky
Summary:
Sometimes kill was exiting with non-zero exit code. Lets make the test machinery kill ncat.

Also during stress runs, we couldn't reuse ports. Let's just randomly allocate them. The test suite does the same thing. but in a different range.

Reviewed By: HarveyHunt

Differential Revision: D33818127

fbshipit-source-id: d875495d590bf8247374ba59396e7032acbdd351
2022-01-28 04:19:00 -08:00
Jan Mazur
8c3c531f2c connect directly to mononoke - remove hgcli
Summary:
I'm:

* adjusting some leftover tests in `fbcode/eden/scm/tests` to speak to mononoke directly
* deleting hgcli from dependencies in `fbcode/eden/mononoke/tests/integration`
* deleting hgcli from dependencies in `fbcode/eden/scm/tests`
* deleting some tests from `fbcode/eden/hg-server/tests`
* removing hgcli usage form scripts like `library.sh` `dummyssh` and `unittestify.pt`

I'm leaving (for now):
* hgcli code and buck rule
* mononoke hgcli codepath

```
➜  tests pwd
/home/mzr/fbcode/eden/scm/tests
➜  tests grep -Ri hgcli
➜  tests cd ~/fbcode/eden/mononoke/tests/integration
➜  integration grep -Ri hgcli | wc
      0       0       0
```

Reviewed By: HarveyHunt

Differential Revision: D33765124

fbshipit-source-id: 46798452f74d59c873449fc6b00445c9558d3ae1
2022-01-28 01:21:32 -08:00
Muir Manders
847a1651f0 rage: limit x2pagent log size on Windows
Summary:
Use the _tail() function which can limit size and has some other smarts.

Also, tweak some progress spinner messages to show up better. "collecting information" was too long and you only saw "collecting". The sub-section spinners "collecting {something}" were also too long and you also only saw "collecting".

Reviewed By: quark-zju

Differential Revision: D33830014

fbshipit-source-id: cb3693c7cb52ac2d060a2da400c9f21ba03f9048
2022-01-27 16:10:16 -08:00
Muir Manders
66a23e61e5 edenapi: let hg-http handle cert config
Summary:
This gets rid of some config boilerplate and allows for centralized cert checks in the http client object.

I left the subtle cert validation checks as-is. We will revisit these later.

Reviewed By: DurhamG

Differential Revision: D33717226

fbshipit-source-id: e9787f7068e5988b83d7c18b60d15ac6eae3c72c
2022-01-27 13:32:13 -08:00
Muir Manders
8dafbc0d42 revisionstore: let hg-http handle lfs cert config
Summary: This gets rid of some config boilerplate and allows for centralized cert checks in the http client object.

Reviewed By: DurhamG

Differential Revision: D33717225

fbshipit-source-id: c3baf271ac0bc56b1c21eab3fd363bccd7dcff37
2022-01-27 13:32:13 -08:00
Muir Manders
47913196d5 hg-http: support configuring cert settings
Summary: This allows TLS config to flow through the http client object rather than being manually configured by each http user.

Reviewed By: DurhamG

Differential Revision: D33717227

fbshipit-source-id: e8f2a49b81f5d30aaedce3517bad8630fb770dfc
2022-01-27 13:32:12 -08:00
Muir Manders
ffbcc29ebf hg-http: migrate rest of HgHttpConfig
Summary:
Move the remaining hg_http::HgHttpConfig fields into http_client::Config. hg_http::http_config() now can derive these config fields from the hg config.

This change cause HTTP requests via EdenFS to:
1. Support x2pagentd unix socket proxy
2. Set X-Client-Info header
3. Respect the http.verbose hg config item to dump request details

Previously the above config items were only set by hg.

Reviewed By: quark-zju

Differential Revision: D33148772

fbshipit-source-id: 4359399aa2d915a2e833fa9ebf8c8860372f0810
2022-01-27 13:32:12 -08:00
Muir Manders
a144039f9b backingstore: conditionalize use of "fb" feature
Summary: Don't set the "fb" cargo feature via cmake unless we are in fb build.

Reviewed By: quark-zju

Differential Revision: D33781554

fbshipit-source-id: 39c61dc1631ebfd8a2c858b5e214c6a09ce0b005
2022-01-27 13:32:12 -08:00
Alex Hornby
e7156fd760 remove unused fb303 dependency
Summary: fb303_core is used. fb303 is not, so remove it

Reviewed By: HarveyHunt

Differential Revision: D33815465

fbshipit-source-id: 149ef2cc48e2eed17fc859707300db555ea5b6b4
2022-01-27 09:41:40 -08:00
Jan Mazur
d4bdd05d60 print testtmp dir before test run only if keep-tmpdir passed
Summary:
This is especially useful when there is long-running test or a test that's stuck. With this path it will be possible to at least look at logs or configs that are being used in the test environment.

I also considered outputting it by default if isatty() == True, but some folks might not like that and it might be too noisy.

Reviewed By: yancouto

Differential Revision: D33792360

fbshipit-source-id: 5c440c3be63242791d92b0decb523c70f4527ee9
2022-01-27 03:23:04 -08:00
Muir Manders
a0dd74742e Back out "print testtmp dir before test run"
Summary:
This breaks the output of mercurial integration tests, e.g.:

```
test-run-tests.t ...
 --- test-run-tests.t
+++ test-run-tests.t.err
@@ -59,6 +59,7 @@

   $ touch test-empty.t
   $ rt
+  testtmp dir: /private/var/folders/w7/8zttmkbx5v94zqfmg4p9kh30000xbj/T/hgtests.1m__f94n/child1/test-empty.t
```

Original commit changeset: 6d131266acfe

Original Phabricator Diff: D33751471 (b1b8a5218a)

Reviewed By: DurhamG

Differential Revision: D33771814

fbshipit-source-id: 7abbba4a5667af29138dfeca9aebb20e162142f6
2022-01-25 12:23:51 -08:00
Andres Suarez
edd73c3cb8 Update tokio to 1.15
Reviewed By: Imxset21

Differential Revision: D33759920

fbshipit-source-id: 7fd02dc93a7c1c564454a1fece3ff4b12dba3d3e
2022-01-25 02:05:10 -08:00
Jun Wu
ad5a787741 match: distinguish gitignore files and directories
Summary:
Fixes a XXX comment. I think it was there because there is a code path that
incorrectly uses matchfn instead of visitdir for directories. But now tests
are passing. The problematic code path seems to have been resolved by
D17099990 (b7680284ca), which tests `kind == dirkind` and then uses `dirignore` or `ignore`
correctly.

Reviewed By: DurhamG

Differential Revision: D33702579

fbshipit-source-id: a06d6a661a552996d90cbe96dd75cb3d4ec5c4d9
2022-01-24 20:24:22 -08:00
Jun Wu
d975e1f900 git: fix auto pull commit hashes
Summary:
Auto pulling commit hashes does not actually work. Fix it and add a test.

Note: This pulls the commit hash as a "draft" if it's based off selected
branches. If we have "public visibleheads" we might use that instead.

Reviewed By: DurhamG

Differential Revision: D33698406

fbshipit-source-id: eac5e4a5536e2c639dee6c7f8b295388b33d26f7
2022-01-24 20:24:22 -08:00
Jun Wu
d7e594deb9 git: drop unused parameter in initgitbare
Summary:
The `url` parameter is no longer used since we maintain git remote config in
hgrc and no longer calls `git remote add`.

Reviewed By: DurhamG

Differential Revision: D33698407

fbshipit-source-id: db891b4370d1b5a42783930c5bda540c799f7211
2022-01-24 20:24:22 -08:00
Jun Wu
49fe26baa2 tests: move git env reset to git.sh
Summary:
It seems there are other tests using `run-tests.py` that do not want the git
env reset behavior. Let's move the env reset logic to a separate optional script.

Reviewed By: DurhamG

Differential Revision: D33752371

fbshipit-source-id: 9465f251ec410595652f1335d1017974876953d7
2022-01-24 17:23:34 -08:00
Jan Mazur
b1b8a5218a print testtmp dir before test run
Summary: This is especially useful when there is long-running test or a test that's stuck. With this path it will be possible to at least look at logs or configs that are being used in the test environment.

Reviewed By: DurhamG

Differential Revision: D33751471

fbshipit-source-id: 6d131266acfef04b0aedad09eae8e5165175705f
2022-01-24 16:15:35 -08:00
Muir Manders
cb62f8d38b addremove: handle symlink shadowing of removed files
Summary: Previously if a removed file was shadowed through a new symlink, addremove would get confused and try to add back the removed file. Fix by "auditing" removed files before we categorize them as "forgotten" (forgotten files are added by addremove).

Reviewed By: DurhamG

Differential Revision: D33748793

fbshipit-source-id: e118baab18b7117d08d87ffb9119530ecdb21acc
2022-01-24 15:15:53 -08:00
Durham Goode
2e29fdfd5d merge: remove O(repo) behavior in _filesindirs
Summary:
When a directory is replaced with a file, Mercurial does some extra
checks. Those extra checks invoke merge._filesindirs() which currently iterates
over every file in the manifest.

This diff uses manifest.matches() to do it way faster.

Reviewed By: danielocfb

Differential Revision: D33722132

fbshipit-source-id: 229e0273f07af71728109db132c6ec72d6ac3dd9
2022-01-24 14:07:30 -08:00
Durham Goode
c34c1c8497 edenapi: cache converted pem files
Summary:
On Windows we're reconverting certs for every request. Let's cache
them.

I thought about trying to store the cache on the client or something, but it
seemed more straight forward and fairly safe to just have a global cache.

Reviewed By: quark-zju

Differential Revision: D33690775

fbshipit-source-id: d3216f5403ea2912f3e49a37aad154d22922487c
2022-01-24 13:44:31 -08:00
Jun Wu
cbd69ed217 test-git-tags: fix for osx
Summary:
There is a name collision `a` vs `A` on OSX. Fix it by reusing the lower-case
name.

Reviewed By: DurhamG

Differential Revision: D33749197

fbshipit-source-id: 27249632f5cc5621b47208ad95c68ca62269be85
2022-01-24 13:44:30 -08:00
Jun Wu
89edf0b94d doctest: fix doctest for osx
Summary: This wasn't discovered because OSX `normpath` path isn't exercised on Linux.

Reviewed By: DurhamG

Differential Revision: D33746541

fbshipit-source-id: 86ccd82477d049208f932eda744a94c03c0d1d02
2022-01-24 13:32:33 -08:00
Jun Wu
6ef8c2c473 test-git-submodule: fix for osx
Summary: `find` without parameter does not work on osx.

Reviewed By: DurhamG

Differential Revision: D33746542

fbshipit-source-id: 9a3665a9d4b25dfe61e72e61e27de36db8f7691c
2022-01-24 13:32:33 -08:00
Jun Wu
f49a2fbc86 config: update tests
Summary: The test was made pass for D33352777 (cf612fcd26) but didn't update for D33518822 (b897a19073).

Reviewed By: DurhamG

Differential Revision: D33746543

fbshipit-source-id: 6e3655a1c1806f6a177b1422b7800f56a5ddac17
2022-01-24 13:32:32 -08:00
Jun Wu
d4d662b828 runtests: do not show # ignored not retesting
Summary:
The message will be repeated for O(tests) times in hgbuild output and makes it
harder to find the actual error message.

Reviewed By: DurhamG

Differential Revision: D33745363

fbshipit-source-id: 67054c9d984c2c07922fc1e82fdec6f55ebf4eb2
2022-01-24 13:32:32 -08:00
Jun Wu
11c3c6d43b setup: upgrade Cython to 0.29.26
Summary: The new version has Python 3.9 support.

Differential Revision: D33744732

fbshipit-source-id: c2ef9c730f1b61e87014f7b01916b2a20b6f68aa
2022-01-24 13:32:32 -08:00
Katie Mancini
b874404b56 bump rust version for eden_apfs_mount_helper
Summary:
the mount helper has a security vulnerability on older versions of rust

let's set the minimum version of rust to use

Reviewed By: fanzeyi

Differential Revision: D33721467

fbshipit-source-id: de78ad38d0b70c236c10c83c4a97c08464b97832
2022-01-21 17:59:51 -08:00
Jan Mazur
f0bda77d8d use mononoke direct connection wherever possible
Summary: hgmn will connect directly to mononoke by default. There are some cases when it's not possible yet. These tests have `NO_MONONOKE_DIRECT_PEER=1` added and need to be adjusted in a more complex way.

Reviewed By: HarveyHunt

Differential Revision: D33683117

fbshipit-source-id: 61fe2c66a3fdcfebd9f5efba27d4de43e38b515f
2022-01-21 03:48:29 -08:00
Durham Goode
9e405ca1e0 revert: don't backup symlinks
Summary:
When doing a revert, Mercurial keeps a copy of reverted files in the
.hg/origbackups directory. If we put symlinks in this directory it can mess up
future file backups, since they might accidentally be put inside the symlink
(which can result in a failure if the symlink is to another device).

Since these backups are almost never used, and are relatively janky, let's just
never backup symlinks.

Differential Revision: D33689167

fbshipit-source-id: 0b227b84122c84e7ea1d12e5219e6b6946c882af
2022-01-20 18:01:59 -08:00