Commit Graph

3587 Commits

Author SHA1 Message Date
Jun Wu
bc7c3c1e24 doctor: remove invalid commits in visibleheads
Summary:
See the test change. Without this change repairing the changelog won't give the
user back a working repo.

Reviewed By: markbt

Differential Revision: D19864421

fbshipit-source-id: b84582c5302469828c8cfcb3db362ea82f2eea63
2020-02-18 09:41:31 -08:00
Jun Wu
febe96048e doctor: integrate fixcorrupt logic to repair changelog
Summary:
Reuse utilities in the fixcorrupt extension to repair changelog.

This is better than fixcorrupt because `hg doctor` does not require a repo
object. Some messages are updated so they become more consistent with the
rest of `hg doctor`.

The main motivation is to get changelog fixed early, so other repair logic can
check if a commit hash is known by changelog or not.

Reviewed By: markbt

Differential Revision: D19864418

fbshipit-source-id: 6f95c6c6191d7db2a474a07a5278a857cf41d8e2
2020-02-18 09:41:31 -08:00
Jun Wu
6f2b529e94 doctor: run 'edenfsctl doctor' on edenfs repos
Summary:
Run 'edenfsctl doctor' on an edenfs repo. If there is no current repo, it might
be caused by edenfs daemon stopped running. So let's also run edenfsctl doctor
in that case.

Reviewed By: markbt

Differential Revision: D19864419

fbshipit-source-id: d2a49a126a040845b88b4883d214162326d08d8d
2020-02-18 09:41:30 -08:00
Durham Goode
99a829340b py3: move sampling to use mercurial.json
Summary:
We're seeing a user have issues because their username contains unicode
characters and sampling's use of json doesn't handle it well. I've not been able
to repro it unfortunately, but let's go ahead and switch sampling to use
mercurial.json.

Differential Revision: D19895419

fbshipit-source-id: a1f087d1e2c7568488c2b8d54f267bd5c8266202
2020-02-18 09:36:28 -08:00
Xavier Deguillard
d8064b5e2a types: add a Sha256 type
Summary: This will be used in the LFS store.

Reviewed By: DurhamG

Differential Revision: D19895803

fbshipit-source-id: 4cf447987c10fed0b5c98904f20c841428965d89
2020-02-18 08:32:33 -08:00
Xavier Deguillard
17cc9ab5ab revisionstore: add a wrapper around IndexedLog/RotateLog
Summary:
In some cases, higher level stores may want to store data in either a plain
IndexedLog, or in a RotateLog, for local and shared data. Due to slight
difference between the 2, they can't easily be adapted into a common trait.

Instead let's just wrap both into an enum and implement the main functions that
the higher level stores need.

The first use of this will be the LfsStore, future use will include the
IndexedLogDataStore and the IndexedLogHistoryStores.

Reviewed By: DurhamG

Differential Revision: D19859292

fbshipit-source-id: 920572e0cf5f69bda4901a727a6b0dc0f08fc8d0
2020-02-18 08:32:32 -08:00
Genevieve Helsel
9b8960eef4 add success field to daemon start logging
Summary: records if a start was successful or not

Reviewed By: simpkins

Differential Revision: D19817810

fbshipit-source-id: b67253099781bb534b7e2fb26a09ba41c1f0bd69
2020-02-18 08:05:51 -08:00
Genevieve Helsel
9c03c5a769 log when a graceful restart is requested
Summary: log when a graceful restart is requested

Reviewed By: simpkins

Differential Revision: D19826640

fbshipit-source-id: 0e2b4ccb36d695a5687386235c8d707461b6e4f3
2020-02-18 08:05:50 -08:00
Genevieve Helsel
175efb9fd1 log when CLI sends sigkill during stop
Summary: Since we cannot log this case from the daemon because we can't catch sigkill, log failed stop from CLI layer.

Reviewed By: simpkins

Differential Revision: D19826140

fbshipit-source-id: eb3aa27802db0206a13e552c4cb1384f856905d2
2020-02-18 08:05:50 -08:00
Genevieve Helsel
c304e34c2d cli scuba logging setup
Summary:
this is used up the stack. This introduces generic scuba logging for the cli layer. In case of the open source build, `log` will be a no-op as suggested in `cli/telemetry.py`.

this is used as so:
```
from .telemetry import build_base_sample, log
# for example, I am adding the field "status" to know that this is a status call.
sample = instance.build_sample("status").add_string("something", "another")
instance.log(sample)
```

Reviewed By: simpkins

Differential Revision: D19816913

fbshipit-source-id: b055d4d1e29456e3549292e6f5047b935f11e4e2
2020-02-18 08:05:49 -08:00
Doug Neal
8e684cfda7 mononoke: lfs_server: add jitter field to ratelimit struct
Summary: Add the max_jitter_ms field to the rate limiting config struct, and to the integration test.

Reviewed By: HarveyHunt

Differential Revision: D19905068

fbshipit-source-id: b44251c456a45bc494d1080e405f2d009becc0d2
2020-02-18 07:47:09 -08:00
Thomas Orozco
49808a4410 mononoke/hg_sync_job: use 0.2 runtime
Summary:
This is required for 0.2 timers or runtime reliant code to work within the sync
job. To achieve this, we need to get of Tokio 0.1 fs code, which is
incompatible with Tokio 0.2 because it uses `blocking()`.

Reviewed By: ikostia

Differential Revision: D19909434

fbshipit-source-id: 58781e858dd55a9a5fc10a004e8ebdace1a533a4
2020-02-18 07:42:41 -08:00
Thomas Orozco
b451a97878 mononoke/warm_bookmarks_cache: use the repo's configuration
Summary:
This update the warm_bookmarks_cache's constructor to use the passed in
blobrepo's derived data configuration (instead of whatever the caller is
passing in), since we now have that information.

Reviewed By: HarveyHunt

Differential Revision: D19949725

fbshipit-source-id: 575a1b9ff48f06003dbf9e0230b7cca723ad68f5
2020-02-18 07:40:25 -08:00
Mateusz Kwapich
69089c1c57 mononoke: Add hash::GitSha1 as a pure hash-only key for git Aliases
Summary: Add hash::GitSha1 as a pure hash-only key for git Aliases, so one no longer needs to know the size or type to load by Alias::GitSha1.

Reviewed By: krallin

Differential Revision: D19903578

fbshipit-source-id: bf919b197da2976bf31073ef04d12e0edfce0f9b
2020-02-18 05:02:52 -08:00
Mateusz Kwapich
f6e5098f7b mononoke: rename GitSha1 to RichGitSha1
Summary:
Rename GitSha1 to RichGitSha1 in preparation for introducing hash::GitSha1 as a pure sha1 without extra fields in next in stack.

Motivation for this is that currently one can't load content aliased by Alias::GitSha1 give just the hash, one has to know the type and size as well.

Once the next couple stack are done we will be able to load via just the git hash.

Reviewed By: krallin

Differential Revision: D19903280

fbshipit-source-id: ab2b8b841206a550c45b1e7f16ad83bfef0c2094
2020-02-18 05:02:51 -08:00
Thomas Orozco
cf4d9c9f1f mononoke/fastreplay: fix off-by-one in load tracking
Summary:
When max concurrency is 1, we should process at most one request concurrently,
not 2!  This had resulted in a flaky test since we're processing traffic out of
order there.

Reviewed By: HarveyHunt

Differential Revision: D19948594

fbshipit-source-id: 00268926095fdbbfdfd5a23366aafcfb763580f4
2020-02-18 04:24:11 -08:00
Liubov Dmitrieva
1e50a66174 prefix look up - use fast path for the full hashes
Summary:
It would be better to make the underlying implementation faster for full hash
cases than check when it is used.

Reviewed By: krallin

Differential Revision: D19905033

fbshipit-source-id: 2d9a77099dc614e80fdb1c0ee715c576a56ba09c
2020-02-18 04:15:03 -08:00
Thomas Orozco
182b50f25e mononoke/apiserver: run Mononoke code on a tokio-compat runtime
Summary:
Right now, Mononoke code in apiserver executes on Actix's runtime. That's a 0.1
runtime, which means that we're calling into code that might just fail if e.g.
it uses Tokio 0.2 timers.

This is a pretty big footgun, so let's fix it. As it turns out, we already have
a Tokio compat runtime in process, which is (was — this is mostly in SCS now)
used for Thrift calls.

So, let's use that runtime to call into Mononoke code. This ensures we don't
get any nasty surprises of the panicky kind at runtime.

Reviewed By: markbt

Differential Revision: D19902538

fbshipit-source-id: d9d7307b8cf75c3e7e1ecf04c0e10076b3eaef3d
2020-02-18 01:55:01 -08:00
Thomas Orozco
16384599a8 mononoke (+ rust/shed/async_unit): update async_unit to expect async fn's
Summary:
This allows code that is being exercised under async_unit to call into code
that expects a Tokio 0.2 environment (e.g. 0.2 timers).

Unfortunately, this requires turning off LSAN for the async_unit tests, since
it looks like LSAN and Tokio 0.2 don't work very well together, resulting in
LSAN reporting leaked memory for some TLS structures that were initialized by
tokio-preview (regardless of whether the Runtime is being dropped):
https://fb.workplace.com/groups/rust.language/permalink/3249964938385432/

Considering async_unit is effectively only used in Mononoke, and Mononoke
already turns off LSAN in tests for precisely this reason ... it's probably
reasonable to do the same here.

The main body of changes here is also about updating the majority of our
changes to stop calling wait(), and use this new async unit everywhere. This is
effectively a pretty big batch conversion of all of our tests to use async fns
instead of the former approaches. I've also updated a substantial number of
utility functions to be async fns.

A few notable changes here:

- Some pushrebase tests were pretty flaky — the race they look for isn't
  deterministic. I added some actual waiting (using pushrebase hooks) to make
  it more deterministic.  This is kinda copy pasted from the globalrev hook
  (where I had introduced this first), but this will do for now.
- The multiplexblob tests don't work at all with new futures, because they call
  `poll()` all over the place. I've updated them to new futures, which required
  a bit of reworking.
- I took out a couple tests in async unit that were broken anyway.

Reviewed By: StanislavGlebik

Differential Revision: D19902539

fbshipit-source-id: 352b4a531ef5fa855114c1dd8bb4d70ed967dd55
2020-02-18 01:55:00 -08:00
Durham Goode
d15cb231fb py3: get fastannotate closer to passing
Summary: There's still some issues, but it's a lot closer.

Reviewed By: quark-zju

Differential Revision: D19802023

fbshipit-source-id: da539094cbc0ba3542e4b5fd3d49f5f80455ec23
2020-02-17 14:52:40 -08:00
Durham Goode
09de146878 py3: fix archive
Reviewed By: quark-zju

Differential Revision: D19802024

fbshipit-source-id: 714f1a3af7f62c8215310fe95ac4de28f9b10f1d
2020-02-17 14:52:40 -08:00
Durham Goode
c9b1fee40f py3: fix eol
Reviewed By: markbt

Differential Revision: D19802025

fbshipit-source-id: f6e6c03ea2d64500b92bcf8b97c9ddedb83fd3c0
2020-02-17 14:52:40 -08:00
Durham Goode
c4f7d32501 py3: enable more tests
Summary: Not sure what fixed these, but let's enable them.

Reviewed By: quark-zju

Differential Revision: D19800696

fbshipit-source-id: 2536a121ae37d4b23ef09dcfee8d92ccbbef3af6
2020-02-17 14:52:39 -08:00
Durham Goode
cd8e154e36 py3: fix test-bookmarks.t
Reviewed By: quark-zju

Differential Revision: D19800698

fbshipit-source-id: a9267e3bafa6f8abb472c0002815fdd2bf2024ca
2020-02-17 14:52:39 -08:00
Durham Goode
779b0beb0b py3: fix test-fb-hgext-phabstatus.t
Reviewed By: quark-zju

Differential Revision: D19800700

fbshipit-source-id: b9b052b5ad1bfa1c714b880cf79c9a21cfebadc2
2020-02-17 14:52:39 -08:00
Durham Goode
6b84a5da81 py3: fix stablerev tests
Reviewed By: quark-zju

Differential Revision: D19800695

fbshipit-source-id: a08efe3c43b1fa01be15ec43a6e32f68dd8eec71
2020-02-17 14:52:38 -08:00
Durham Goode
a6353c55f5 py3: fix sparse
Reviewed By: quark-zju

Differential Revision: D19800699

fbshipit-source-id: 7cf86d0ab9a0efc96966ac3747f56b229251fb0d
2020-02-17 14:52:38 -08:00
Durham Goode
00d72e96bf py3: fix gitlookup
Reviewed By: xavierd

Differential Revision: D19800701

fbshipit-source-id: ecc918a0ef7358a29789f0ab6ca50337bec9edf1
2020-02-17 14:52:38 -08:00
Durham Goode
f05a9d79bb py3: fix test-flagprocessors.t
Reviewed By: xavierd

Differential Revision: D19800697

fbshipit-source-id: a785a4a04ce627257993d83197fec270a8688b7d
2020-02-17 14:52:37 -08:00
Durham Goode
fdbaaeeff9 py3: fix treedirstate unicode handling
Summary:
There was a spot where we returned bytes for a filepath. Fix this to
make dirstate tests pass more.

Reviewed By: quark-zju

Differential Revision: D19786274

fbshipit-source-id: 7465cae8bb2e3be7758abc6279ed3f5f59581732
2020-02-17 14:52:37 -08:00
Durham Goode
4f8c30b04e py3: enable 92 py3 tests
Summary: These now pass

Reviewed By: singhsrb

Differential Revision: D19785175

fbshipit-source-id: bf92757e3fe0753e1b61ffddfd30a37fb40a642f
2020-02-17 14:52:37 -08:00
Durham Goode
c2f42204cc py3: fix some mutation tests
Reviewed By: quark-zju

Differential Revision: D19772615

fbshipit-source-id: 93bcefb6d911941a9e46107f124730279398ceff
2020-02-17 14:52:36 -08:00
Durham Goode
5f46843003 py3: get several infinitepush tests passing
Reviewed By: quark-zju

Differential Revision: D19772622

fbshipit-source-id: c0b64e3a22c085656dbff5526765020b64cc73f5
2020-02-17 14:52:36 -08:00
Durham Goode
5827145ef2 py3: fix some infinitepush py3 issues
Reviewed By: quark-zju

Differential Revision: D19772618

fbshipit-source-id: c5e520a3cec6496e24cc5b3825ef7287ab82f304
2020-02-17 14:52:36 -08:00
Durham Goode
56f4d5ae18 py3: enable a few tests that got fixed along the way
Summary: Not sure where these got fixed.

Reviewed By: quark-zju

Differential Revision: D19772617

fbshipit-source-id: 7bebd15ad080e1fc224d8c1e78f645877551ac92
2020-02-17 14:52:35 -08:00
Durham Goode
2c18b7521a py3: make test-fb-hgext-remotefilelog-clone-tree.t pass
Reviewed By: quark-zju

Differential Revision: D19772620

fbshipit-source-id: 26d1ae840e1e19c7dabbdf7a71e2d4b0d265e141
2020-02-17 14:52:35 -08:00
Durham Goode
255b677f91 py3: fix test-fb-hgext-remotefilelog-lfs.t
Reviewed By: quark-zju

Differential Revision: D19772621

fbshipit-source-id: 25fda0f356129b27413b25cfb70e9d08bbc56263
2020-02-17 14:52:35 -08:00
Durham Goode
a8b4789109 py3: fix misc encoding errors that showed up in tests
Summary:
These are a random batch of stack traces that show up in remotefilelog
tests.

Reviewed By: quark-zju

Differential Revision: D19772619

fbshipit-source-id: a8b9ce188cb7a5a2c9ccaeb62f6744f1c4083e38
2020-02-17 14:52:34 -08:00
Durham Goode
2b0c3ec119 py3: fix test-fb-hgext-remotefilelog-histpack.py
Reviewed By: quark-zju

Differential Revision: D19772616

fbshipit-source-id: 443527d37aa87c1802323aa6edf863beb497a327
2020-02-17 14:52:34 -08:00
Durham Goode
a5ba21d4d4 py3: codemode assertEquals to assertEqual
Summary:
assertEquals is deprecated and shows warnings in the tests in python 3

ignore-conflict-markers

Reviewed By: quark-zju, sfilipco

Differential Revision: D19907385

fbshipit-source-id: 7d261489856a4eeb6719eae581ed986d0415d99e
2020-02-17 14:52:34 -08:00
Durham Goode
41cc67d011 py3: fix remotefilelog repack tests
Reviewed By: quark-zju

Differential Revision: D19751790

fbshipit-source-id: a898a8c37929a73ee2654d2907e9ea5bb2e3fd8a
2020-02-17 14:52:33 -08:00
Durham Goode
3a5728b32f py3: fix archive
Summary:
archive uses a formatter to produce it's metadat file. We need to use a
string io stream instead of bytes.

Reviewed By: quark-zju

Differential Revision: D19748163

fbshipit-source-id: fbd8c32066cfc4a234d9b51691717c7fce4c7c9a
2020-02-17 14:52:33 -08:00
Durham Goode
48aec8ca41 py3: misc fixes to remotefilelog and treemanifest
Reviewed By: quark-zju

Differential Revision: D19747715

fbshipit-source-id: da6716ff46342d777d45bec8e560ab41d544645f
2020-02-17 14:52:33 -08:00
Durham Goode
150aa7be3d py3: fix bundle2 bookmark default values
Summary: They need to be bytes to match the nodes.

Reviewed By: xavierd

Differential Revision: D19746027

fbshipit-source-id: 41e9cc390f4aa97d8c8b378144e64100811665c2
2020-02-17 14:52:32 -08:00
Durham Goode
6d40f2c380 py3: fix occasional ssh hang
Summary:
sshpeer._calltwowaystream used iter(fp, ""), where "" was the sentinel
indicating the fp was empty. Since fp was an iterator of bytes, the sentinel was
never hit and this code kept sending 0\n to each other until the pipes filled
and it hung.

Reviewed By: xavierd

Differential Revision: D19746028

fbshipit-source-id: 0daada0ae2356b5c99cc2c39c121cd88af8f750a
2020-02-17 14:52:32 -08:00
Durham Goode
a3ec80f7f5 py3: various fixes to move bundle2 tests foward
Reviewed By: quark-zju

Differential Revision: D19717253

fbshipit-source-id: 3f07a3c6ff09c1f7081d2dd15659505b4a07a91a
2020-02-17 14:52:32 -08:00
Durham Goode
67ade10253 py3: fix minor bookmark and sparse encoding issues
Reviewed By: quark-zju

Differential Revision: D19717254

fbshipit-source-id: 1ee7fab0bec1bb5b42c2707e7bc07aa8300f4be7
2020-02-17 14:52:31 -08:00
Durham Goode
4d4274ce53 py3: make conduithttp.py python 3 compatible
Reviewed By: quark-zju

Differential Revision: D19717252

fbshipit-source-id: 075aaed642898a3649d40f9cdc9752539b0970aa
2020-02-17 14:52:31 -08:00
Durham Goode
66fd50346c py3: add pycompat buffer alias
Summary: buffer in Python 2 became memoryview in Python 3.

Reviewed By: quark-zju

Differential Revision: D19717251

fbshipit-source-id: f678abfed7e2ec94053329b13015bff3c31eb319
2020-02-17 14:52:30 -08:00
Durham Goode
e4b5247311 py3: fix test-cat.t
Summary:
Fixes test-cat.t by changing some formatter encoding choices. The
formatter is bit awkward here.  hg cat needs to support outputting raw binary,
but also json and templated output.

For now I've set it up so json and templated output can't output non-utf8 data.

Reviewed By: quark-zju

Differential Revision: D19786542

fbshipit-source-id: 84060928103b396b23e3173b715aed996074fa3e
2020-02-17 14:52:30 -08:00