Commit Graph

45292 Commits

Author SHA1 Message Date
Jun Wu
72febd6068 test-remotenames-schemes: rewrite the test
Summary:
The test uses named branches heavily. It seems the test just wants to make
sure remotenames work with the schemes extension. Rewrite it to stop using
named branches.

Reviewed By: singhsrb

Differential Revision: D13978570

fbshipit-source-id: 3d0f284bb2398da0283b3bf5a06d879182a70d84
2019-02-12 19:54:30 -08:00
Xavier Deguillard
25f6272a15 test-fb-hgext-treemanifest-repack.t: use packsonly when purging packs
Summary:
Purging of packs may happen before the repack operation, and when loosefiles
are also present, repack will reconstruct the packs from them. By forcing a
packsonly, we won't have a chance of reconstructing the purged ones.

Differential Revision: D14013788

fbshipit-source-id: d77d9ec2b9c5a086caaf9164a8ac56dd7d1e1255
2019-02-12 11:21:34 -08:00
Xavier Deguillard
e241404290 remotefilelog: always refresh before runonpacks
Summary:
The refresh function is intended to load on-disk packfiles that aren't yet
present in the cache, but it had several issues that resulted in unexpected
behavior. Mainly, the new files would be added in the wrong order in the cache,
and runonpacks would not update the last used packs.

Differential Revision: D14013785

fbshipit-source-id: da4151b859441f0ba51863f3907852922c255f82
2019-02-12 11:21:34 -08:00
Xavier Deguillard
f5d7209d39 remotefilelog: move rustrepack
Summary:
Previously, the logic for chosing between the rust repack, and the python one
was deeply integrated with the python repack. This resulted in work being
performed unnecessarily.

The drawbacks of this method is that in the case of fallback to Python, the
repacklock will be dropped and re-obtained, as well as the prerepack hook will
be ran twice. Since the python code will soon be entirely replaced this is
probably not important.

Differential Revision: D14013790

fbshipit-source-id: 7c754db3fb984cc9e6b7df1a4e32b72fa4a1531b
2019-02-12 11:21:34 -08:00
Xavier Deguillard
f134a4fe92 remotefilelog: refactor incremental/full repack
Summary: Most of the code is identical between these 2 functions, so let's merge them.

Differential Revision: D14013787

fbshipit-source-id: f0469e05fcf723db02d9f4c28097e61f958284a5
2019-02-12 11:21:34 -08:00
Xavier Deguillard
73aed5c3d2 revisionstore: do not attempt repacking one packfile
Summary:
Repacking one packfile will yield the same packfile, so we can save some IO by
not trying to repack.

Differential Revision: D14013789

fbshipit-source-id: 8069840cc7cb1837eb94cea97e50b3bbaa548873
2019-02-12 11:21:34 -08:00
Jun Wu
946fa43efd fsmonitor: update watchman clock if too many files are returned
Summary:
This should auto recover from a state where watchman returns too many files,
which makes "hg status" slow.

The default config value 200 is not chosen very scientifically, but it should
cover the 6s status case where there are 90k changed files reported.

Differential Revision: D14036494

fbshipit-source-id: 368f53e99e2e54343a6ac9145cbb86e4ac65e4ac
2019-02-12 11:16:12 -08:00
Jun Wu
b53bc49971 fsmonitor: add a test showing suboptimal behavior
Summary:
akushner encountered a case where "hg status" takes 5+ seconds and cannot
self-recover. That is because watchman constantly returns huge number of files
and fsmonitor didn't choose to update the clock to address that.

Reviewed By: DurhamG

Differential Revision: D14036493

fbshipit-source-id: 0bd8f7c9e3a51b6b4d2a1584b68883f8579e3f0f
2019-02-12 11:16:12 -08:00
Jun Wu
d515e4826f config: still load configs in legacy locations
Summary:
D13875656 made a config path change that breaks tests without using HGRCPATH,
or local build runs.

Reviewed By: DurhamG

Differential Revision: D14034919

fbshipit-source-id: 80de214f1769a8f40e79dc0ab1dbba4d55f506a7
2019-02-11 17:08:45 -08:00
Arun Kulshreshtha
cd9197c25d revisionstore: fix import ordering
Summary:
We've settled on the following grouping for imports:

- standard library
- 3rd party crates
- internal crates
- modules within same crate

This diff updates revisionstore accordingly.

Reviewed By: singhsrb

Differential Revision: D14030243

fbshipit-source-id: 74a7897342e39eb1d80202c8aae8c149bf08fc41
2019-02-11 15:47:36 -08:00
Jun Wu
98bb16cace hgsql: fix hggit sync
Summary: Teach mysql connector to deal with bytearray.

Reviewed By: DurhamG, StanislavGlebik

Differential Revision: D14025816

fbshipit-source-id: 738d9ef37b985afad5fe62815be134a4a5913b9a
2019-02-11 13:47:12 -08:00
Jun Wu
33de65a309 test-hgsql-hggit: demostrate bytearray encoding error
Summary: This is caused by D13516212, and not covered by tests.

Reviewed By: DurhamG, StanislavGlebik

Differential Revision: D14025817

fbshipit-source-id: 8a529085c40c5d2ae17ccf3de1ccea22c8d7dbd2
2019-02-11 13:47:12 -08:00
Kostia Balytskyi
eaab89f103 test-fb-hgext-remotefilelog-corrupt-cache: do not use truncate
Summary: It is unavailable on OSX, but we have a custom implementation.

Reviewed By: DurhamG

Differential Revision: D14028103

fbshipit-source-id: 4b6916792fc9da43795b0f9db57ec2a057c92283
2019-02-11 13:06:29 -08:00
Kostia Balytskyi
c526eed2fa test-profile: fix the edenscm-rename fallout
Summary: Should fix https://fburl.com/sandcastle/3sdblrdd.

Reviewed By: mitrandir77

Differential Revision: D14026757

fbshipit-source-id: 685bc95a13309459a015f4ae93abd500db9eeb11
2019-02-11 11:14:17 -08:00
Arun Kulshreshtha
f7bbff1ceb edenapi: do not enforce HTTP/2 only
Summary: It turns out that `hyper-tls` does not support ALPN for negotiating HTTP/2 connections, and only supports HTTP/2 prior knowledge. (This is a limitation of the underlying TLS library, `native-tls`.) Unfortunately, while the Mononoke API server itself is fine with HTTP/2 prior knowledge for non-TLS connections, the Mononoke VIPs require TLS, and thus per the HTTP/2 spec require ALPN negotiation from an HTTP/1.1 initial connection. As a result, we need to revert back to using HTTP/1.1 for now in order to use TLS.

Reviewed By: singhsrb

Differential Revision: D14015335

fbshipit-source-id: b78197d4cfecf184479162c5b14ba54cbef66ee7
2019-02-11 09:54:22 -08:00
Aida Getoeva
3aee83d26a prefetch: replaced store path with the cache path
Summary:
To connect to the memcache client it used a path from a loose files data store (`remotefilelogcontentstore._path`), which is a path to the hg cache, but pack files store doesn't have the same field.
I replaced the path with cache path from shallowutil.

Reviewed By: singhsrb, liubov-dmitrieva

Differential Revision: D14020122

fbshipit-source-id: 29bc8a01fc42d43eddc6ae67cf4e41fc552e117c
2019-02-11 09:38:43 -08:00
Jun Wu
f656af6866 encoding: alias cp65001 to utf-8 on Windows
Summary:
This back ports Yuya's patch from upstream:

  changeset:   443029011990c75c533ec0454fd0f5c1060d4690
  branch:      stable
  user:        Yuya Nishihara <yuya@tcha.org>
  date:        Sun, 01 Jul 2018 06:36:53 -0800

      encoding: alias cp65001 to utf-8 on Windows

      As far as I can tell, cp65001 is the Windows name for UTF-8. I don't know
      how different it is from the UTF-8, but Python 3 appears to have introduced
      new codec for cp65001, so the alias is enabled only for Python 2.

      https://bugs.python.org/issue13216

      This patch is untested, but hopefully fixes the following issue.

      https://bitbucket.org/tortoisehg/thg/issues/5127/

More context:
Windows 10 Build 17035 (November 2017) introduced a
"Use Unicode UTF-8 for worldwide language support" checkbox, which makes the
"A" flavored APIs use UTF-8 instead of some non-UTF-8 local encoding. With
that checkbox ticked, hg would crash in cmd.exe complaining
"LookupError: unknown encoding: cp65001"

{F150484107}

Reviewed By: phillco

Differential Revision: D14017361

fbshipit-source-id: a648b2a9bc341d796532c6fd2730c083402b3736
2019-02-11 09:16:02 -08:00
Jun Wu
9cdc2640d6 config: change system config entry point
Summary:
Change system config entry point to only `/etc/mercurial/system.rc` (unix) and
`\ProgramData\Facebook\Mercurial\system.rc` (Windows) so they won't overlap
with a vanilla Mercurial installation.

Another goal of this change is to make it easier to drop the directory
`%include` feature. So detecting config changes (for example, edenfs wants to
make sure ignore rules are up-to-date) can be made cheaper by just stating
files without `listdir`.

Reviewed By: markbt

Differential Revision: D13875656

fbshipit-source-id: 314c0bf87ff086dec5b88e232edca0133356484e
2019-02-08 19:31:11 -08:00
Xavier Deguillard
0ce06567ab remotefilelog: properly declare CacheConnectionError
Summary:
The super python builtin expects a type as an argument, while a string was
passed, this was causing trouble for some users.

Reviewed By: DurhamG

Differential Revision: D14012716

fbshipit-source-id: 6714eb20745428818721ec3df85588a7b4c7ebb7
2019-02-08 17:16:00 -08:00
Jun Wu
b74ee0564d chg: make it incompatible with upstream chg server
Summary:
It would be massy if there are 2 chg servers running: one for fb hg, one for
upstream hg, and they share a same socket path.

Change socket path and the commandserver name so fb-hg chg can only talk to
fb-hg chg servers.

Reviewed By: markbt

Differential Revision: D13869319

fbshipit-source-id: f9d42af9bdfc542207f23c536b478fd5ef8d02a0
2019-02-08 16:12:53 -08:00
Jun Wu
495cdac243 test-remotenames-with-aliases: remove the test
Summary:
The test is about treating the "default" remote branch specially.
Named branches are going away, therefore delete the test.

Reviewed By: ikostia

Differential Revision: D13976850

fbshipit-source-id: 0283e78c120efd0cf9c91811b823923780cdbfd9
2019-02-08 14:40:28 -08:00
Jun Wu
a379d745b3 test-merge-default: rewrite part related to named branch
Summary: This makes the test future-proof.

Differential Revision: D13954461

fbshipit-source-id: 373afe1df712856a46638da8ea84cf2e56c494ab
2019-02-08 14:40:28 -08:00
Jun Wu
4021b6f311 test-pull-update: remove part related to named branch
Summary: This makes the test future-proof.

Differential Revision: D13954455

fbshipit-source-id: a98918dff351db2117ed57b852d390312bb203fb
2019-02-08 14:40:28 -08:00
Jun Wu
93867ff88f test-subrepo: remove the test
Summary:
The test uses named branch. Since subrepo is also on the list to kill, just
remove the test now.

Differential Revision: D13954459

fbshipit-source-id: 29e8009376081d964441497796fda3490f2137e3
2019-02-08 14:40:28 -08:00
Jun Wu
b5599c717f test-glog: rewrite part related to named branch
Summary: This makes the test future-proof.

Differential Revision: D13954463

fbshipit-source-id: 40fb1044dbe5bacfc68c6b04dca15f1c3dc92df1
2019-02-08 14:40:28 -08:00
Jun Wu
dec8f9f3e9 test-ssh: rewrite part related to named branch
Summary: This makes the test future-proof.

Differential Revision: D13954457

fbshipit-source-id: f48c2ac7ba5072bae458283f7b3afb4900b78442
2019-02-08 14:40:27 -08:00
Jun Wu
66ea9f2284 test-revset-outgoing: rewrite part related to named branch
Summary: This makes the test future-proof.

Differential Revision: D13954462

fbshipit-source-id: 42031851ecb6be66a2ec61fae7107b8b6f498afc
2019-02-08 14:40:27 -08:00
Jun Wu
7a1977912a test-rebase-pull: remove part related to named branch or tags
Summary: Make it future-proof.

Differential Revision: D13954454

fbshipit-source-id: c6a746ae4035cdff8ead88514aa09f8ef3db0c18
2019-02-08 14:40:27 -08:00
Jun Wu
7637cfd5f3 test-fb-hgext-fastannotate-remotefile: remove the test
Summary:
The test was about the connection handling between remotefilelog and
fastannotate. It uses named branch to test certain code paths, which
is not trivial to migrate off.

We'll probably rework the protocol used here and maybe do something very
different regarding on linelog with Mononoke. Therefore just remove the
test for now.

Reviewed By: singhsrb

Differential Revision: D13934435

fbshipit-source-id: 9ba0b760ba17efd3ede6de7612e6532f3d13a4f8
2019-02-08 14:40:27 -08:00
Jun Wu
b8b6316334 test-convert-splicemap: rewrite part related to named branch
Summary: This makes the test future-proof.

Differential Revision: D13954466

fbshipit-source-id: d8ba31e53eb912451ba451b367225af66c7194ac
2019-02-08 14:40:27 -08:00
Jun Wu
564597d13f test-rebase-check-restore: rewrite part related to named branch
Summary: This makes the test future-proof.

Differential Revision: D13954469

fbshipit-source-id: 94c0289dc7ef8159592bd541da4021c5152dc298
2019-02-08 14:40:27 -08:00
Jun Wu
ca843d3676 test-sparse-merges: rewrite part related to named branch
Summary:
The test was relying on the ability to merge a "fast-forward" commit, it's no
longer possible without named branches. So the DAG was changed from two linear
commits to "Y"-ish 3 commits.

This makes the test future-proof.

Differential Revision: D13954460

fbshipit-source-id: 944747404843e985f06d18fe0618843dca64630f
2019-02-08 14:40:27 -08:00
Xavier Deguillard
31622c9806 asyncpacks: add AsyncUnionHistoryStore
Summary: This will be used by scmmemcache to send history data to memcache

Reviewed By: DurhamG

Differential Revision: D13975346

fbshipit-source-id: f41eaf9a4968072dd07efbcd9d539e6293c3fa4f
2019-02-08 12:56:06 -08:00
Xavier Deguillard
a4311ec1df memcache: implement get_hist
Summary:
We can now fetch history data stored in memcache and write it to a history
pack.

Reviewed By: DurhamG

Differential Revision: D13975308

fbshipit-source-id: 2196328ad60a55d1e2b39d88d939f434e496837a
2019-02-08 12:56:06 -08:00
Xavier Deguillard
d9153f1565 memcache: proper serde serialization
Summary:
The initial get_data/set_data only sent the full-text to memcache, which is
just enough for non-LFS data. Let's use Serde to serialize/deserialize the data
that we send to memcache. This will make it simple to add checksuming, or more
metadata to it.

Reviewed By: DurhamG

Differential Revision: D13974714

fbshipit-source-id: 41a235e1d1e8128b14f00b668745f4f9a070a360
2019-02-08 12:56:06 -08:00
Xavier Deguillard
76418dd79c memcache: add set_data
Summary:
Similarly to the get_data, we can now read a datapack and send the proper
deltas to memcache. This change is lacking in the same way the get_data is.

Reviewed By: DurhamG

Differential Revision: D13886026

fbshipit-source-id: a00475e89b7e75dbbe9afa9f9d293a686f969a3f
2019-02-08 12:56:06 -08:00
Xavier Deguillard
9439d09d10 revisionstore: implement IterableStore for UnionStore
Summary:
The IterableStore trait allows iterating over all the keys of a DataStore.
Since this is applicable to a UnionStore, let's implement it there. We can now
use it in their async variants.

Regarding the async variants, the code effectively builds a Vec of Key, which
may use a lot of memory, a better alternative would be to use a Stream of Key.
This will be tackled later.

Reviewed By: DurhamG

Differential Revision: D13951905

fbshipit-source-id: 15944b18d7ffea08d191e5dc7e1b8e2b783f69d1
2019-02-08 12:56:06 -08:00
Xavier Deguillard
374495767e asyncpacks: add AsyncUnionDataStore
Summary: Simple async wrapper around a UnionDataStore.

Reviewed By: DurhamG

Differential Revision: D13951906

fbshipit-source-id: 086739d834297fcc1dabd246cfde4631b4767640
2019-02-08 12:56:06 -08:00
Stefan Filip
2d6e900d12 Update itertools version in lib/treestate
Summary: TP2 version for itertools was updated to 0.8.

Reviewed By: singhsrb

Differential Revision: D14008855

fbshipit-source-id: 081a43c5b02cd39c6a0a6b491bfa0767ddf0b7ed
2019-02-08 11:49:54 -08:00
Saurabh Singh
b02696d961 remotefilelog: fix the test broken by defaulting file blobs to version 1
Summary: This test was broken by D13980892 and needs to be updated.

Reviewed By: DurhamG

Differential Revision: D14001840

fbshipit-source-id: 6487159445f8aa049996a35e1bb70dad5af614ac
2019-02-08 10:52:16 -08:00
Liubov Dmitrieva
ca7f631006 lazy connection open for memcache has an issue with SIGPIPE
Summary: My earlier diff broke master, this is the fix.

Reviewed By: markbt

Differential Revision: D14005894

fbshipit-source-id: 6ce43913fb501791592512f2f12cd67ef27f0457
2019-02-08 09:44:06 -08:00
Liubov Dmitrieva
261da2a786 fix remotenames for infinitepush
Summary:
Currently if default path and infinitepush both point to the remote path
from the repo, the activepath function returns 'infinitepush' rather than
'default' because of the logic I have fixed.

It makes your smartlog looks like.

```
o  037dddb5  62 minutes ago  infinitepush/fbobjc/stable remote/fbobjc/stable
.
o  66e0e00c  Today at 05:58  infinitepush/fbandroid/stable
remote/fbandroid/stable
.
@  28c3b5b0  Today at 05:40  mlesyk  D14003881  T27597699
.  [fbar][prod_role] fix region detection part
.
o  407504d6  Today at 04:58  infinitepush/fbsource/stable
remote/fbsource/stable
```

When you pull it updates infinitepush/master, etc instead of remote/master,
etc.

This only happens if they both point to the same url.

This is not desirable for infinitepush.

I also introduced infinitepush-other that we will use for the secondary commit cloud backend.

It would have the same issue if it matches the url of the default path.

Reviewed By: markbt

Differential Revision: D14005639

fbshipit-source-id: f3ea8098e1c418e2efd43b5430bb4252f6ad6b2c
2019-02-08 09:24:20 -08:00
Liubov Dmitrieva
3cc2093659 fix tests
Summary: Size is not always the same, let just use number of files.

Reviewed By: rlangst

Differential Revision: D14004261

fbshipit-source-id: 13e21d8fd95ee09fc54c750ceecc48bcd484bc9e
2019-02-08 08:30:11 -08:00
Wez Furlong
35e2b7c6ff hg: fix lfs and journal extension postshare wrapping
Summary:
this fixes `hg-new-workdir` by matching the function signature
and propagating the arguments.

Reviewed By: singhsrb

Differential Revision: D13992348

fbshipit-source-id: 582025affe2e0511f239e82bc880b2fecea84ac7
2019-02-07 18:58:40 -08:00
Wez Furlong
45a59ef1d4 hg: run-tests: treat a parse error as return code 1
Summary:
previously, unexpected output would cause the harness to
stack trace and not show the actual test failure.

This catches the error and treats it as a failed run, which causes
the test output to be displayed.

No more `ValueError: invalid literal for int() with base 10: '0\xe2\x90\x8a'`

Reviewed By: DurhamG

Differential Revision: D13995000

fbshipit-source-id: 5c1a9e8d6f8e81a924cdbf760fa75cdc0d33b74d
2019-02-07 18:58:40 -08:00
Jun Wu
9efa79ce5c test-commit-multiple: rewrite parts related to named branch
Summary:
Change the test to avoid creating branches so it could still work
once `hg branch` gets removed.

Differential Revision: D13954467

fbshipit-source-id: ec6ee3430cecd4ee4daa23b303f0a36be9198659
2019-02-07 18:17:19 -08:00
Jun Wu
5dfbe25ca2 test-mv-cp-st-diff: rewrite parts related to named branch
Summary:
Change the test to not create branches, so it could still work
once `hg branch` gets removed.

Differential Revision: D13954465

fbshipit-source-id: 10ff86b2a765cb8bd57d1589a4e3624d553b4d07
2019-02-07 18:17:18 -08:00
Jun Wu
512dba0175 test-rollback: remove the test
Summary:
The test relies on named branches. Since the "hg rollback" command will also be
removed in the future, just delete the test for now.

There are a couple of other tests relying on the "hg rollback" command, which
makes the "rollback" command itself harder to delete.

Differential Revision: D13954464

fbshipit-source-id: 6a56ca15de96258c79deb90114438b42d802d8b0
2019-02-07 18:17:18 -08:00
Jun Wu
c2ef481658 logexchange: delete the experimental component
Summary:
logexchange was a subset of remotenames, added by:

  changeset:   5a62910948d2d4bac5defe305d0ddb22f0fda549  D1547
  user:        Pulkit Goyal <7895pulkit@gmail.com>
  date:        Wed, 04 Oct 2017 10:32:02 -0800
  summary:     remotenames: move function to pull remotenames from the remoterepo to core

Since we use the full remotenames instead, remove logexchange and its test,
which depends on named branch.

Differential Revision: D13954458

fbshipit-source-id: d565c131100ef90f3cf69e9051643ac8e5846f0d
2019-02-07 18:17:18 -08:00
Jun Wu
141c848e2f test-encoding-align: remove part related to named branch or tags
Summary: Make it future-proof.

Differential Revision: D13954468

fbshipit-source-id: ee0f856c6d424b99f6b7406d12cd1f96012775d5
2019-02-07 18:17:18 -08:00