Commit Graph

63825 Commits

Author SHA1 Message Date
Alex Hornby
44474c8d8a mononoke: log size of object in lfs validation
Summary: Its useful to know the size as well as the fact its LFS

Reviewed By: mitrandir77, farnz

Differential Revision: D26945223

fbshipit-source-id: 42787b983626ceecf822380e8ec6268646b3338f
2021-03-10 06:51:54 -08:00
Alex Hornby
b707609eb3 mononoke: add lfs validate step to report files over lfs threshold
Summary: Add a validate check that can log which files are over the LFS size threshold

Reviewed By: farnz

Differential Revision: D26853691

fbshipit-source-id: 414e608358ae0cf6e3f7f55e21caf253a1dc2f9c
2021-03-10 06:51:53 -08:00
Alex Hornby
f8c5402f22 mononoke: allow validate steps to report resolved repo path
Summary:
Some node types don't hold the path as part of their key (e.g.
FileContent)  but can still have interesting path information associated with the edges to them.

This is used in next diff to add validation that can report on LFS files

Reviewed By: farnz

Differential Revision: D26945222

fbshipit-source-id: b78347bc81fc02fdc3b71a76522b2986c772440a
2021-03-10 06:51:53 -08:00
Alex Hornby
136b659a64 mononoke: allow walker validate Pass to return info for logging
Summary: Shouldn't have to be a failure to log from validation.  This updates FailureInfo to ValidateInfo in preparation for adding an LFS validation check in next diff

Reviewed By: farnz

Differential Revision: D26853692

fbshipit-source-id: 9fbee1e5b31664365a75aa207f055b7880ce326c
2021-03-10 06:51:53 -08:00
Stanislau Hlebik
7a3539b9c6 mononoke: use correct repo name in darkstorm sync job
Summary:
For darkstorm sync job we shouldn't use the source mononoke repo name, because
it breaks our logging and alarms.

This diff fixes it

Reviewed By: farnz

Differential Revision: D26945466

fbshipit-source-id: d90abd0cf2e1c480d529d70f825a14f1460d2e29
2021-03-10 06:02:44 -08:00
svcscm
2596edbee5 Updating submodules
Summary:
GitHub commits:

640db0c915

Reviewed By: yns88

fbshipit-source-id: f573afac1e6e09d548879e556fc9cabd241c44c5
2021-03-10 04:56:29 -08:00
Alex Hornby
efa3f15938 mononoke: force mysql index for phases query by repo_id and cs_id
Summary: Make sure query is always using the index

Reviewed By: farnz

Differential Revision: D26920481

fbshipit-source-id: dfd274f1ff13fc0e689265ff3bd1831e59d3f012
2021-03-10 03:57:49 -08:00
Mark Juggurnauth-Thomas
7ed5a54640 repo_client: remove case conflict checks on upload
Summary:
Remove case conflict checking on upload.  Disallowing case conflicts will
always be done during bookmark movement by checking the skeleton manifests.

A side-effect of this change is that configuring a repository with case
conflict checks, but not enabling skeleton manifests, will mean that commits
can't be landed in that repository, as there are no skeleton manifests to
check.

Reviewed By: ahornby

Differential Revision: D26781269

fbshipit-source-id: b4030bea5d92fa87f182a70d31f7e9d9e74989a9
2021-03-10 03:51:42 -08:00
svcscm
ff0fa0ee31 Updating submodules
Summary:
GitHub commits:

60d12a2ff6
4b18c46d10

Reviewed By: yns88

fbshipit-source-id: ff889890970dade590dae5eca29cc8dad8256614
2021-03-09 20:56:19 -08:00
Jun Wu
1225d154d8 visibility: extract pure serialization to a function
Summary: It'll be reused elsewhere.

Reviewed By: kulshrax

Differential Revision: D26924712

fbshipit-source-id: 0be666a1acd0bdfbc14bd39d4193a06a9929eed7
2021-03-09 18:52:24 -08:00
svcscm
6b994651ef Updating submodules
Summary:
GitHub commits:

6a3688bc53
2332ce8a36

Reviewed By: yns88

fbshipit-source-id: d22ee70aa22f2228bbf5df50306faaf2f3cc227f
2021-03-09 18:52:23 -08:00
svcscm
9d0909ba72 Updating submodules
Summary:
GitHub commits:

e6d7e59233
ee54f661ae
ed8dad312b
0028e3398b
5aeb7d52ed
79e58150e2
eeafd4036a
72ea985334
69371a22a4
06f5f6285b
4ca5d47f5b

Reviewed By: yns88

fbshipit-source-id: 5344ec93023dcf36e0926c698dd8dc72c193d1c5
2021-03-09 13:40:55 -08:00
generatedunixname89002005307016
0f3863c783 suppress errors in fbcode/eden - batch 1
Differential Revision: D26914007

fbshipit-source-id: dc68c762e9c33b481bbd98959579e0c0836f17c3
2021-03-09 13:23:39 -08:00
Jun Wu
3db2121dec async-runtime: add a way to spawn task using the runtime
Summary: This API will be used to spawn tasks in the runtime.

Reviewed By: andll

Differential Revision: D26615607

fbshipit-source-id: 6b9892fad755bbe8feb775e9dad457697b2ea1b7
2021-03-09 11:45:28 -08:00
Thomas Orozco
a8d5625490 mononoke: fix some tests
Summary: Looks like this formatting changed in hg? This is failing on master

Reviewed By: mitrandir77

Differential Revision: D26914873

fbshipit-source-id: 91f3e1684bba1d66a740c5c605c31a8b25e42964
2021-03-09 10:54:00 -08:00
svcscm
420e73d704 Updating submodules
Summary:
GitHub commits:

e04b111971
438afd7d7e
d7d9a555d0

Reviewed By: yns88

fbshipit-source-id: b18fe7100a249c2369d914c5fca3f8e63e2823d6
2021-03-09 10:20:19 -08:00
Thomas Orozco
ae96770bc0 mononoke/fileblob: don't silently ignore write errors
Summary:
Right now, if we get an error when writing to fileblob, we silently ignore it
under the assumption that it's an overwrite that failed. We shouldn't do this,
since it can cause to silently discard data.

For example, when using Mononoke with a tmpdir that isn't next to your blobs,
one might hit this error:

```
Os { code: 18, kind: Other, message: "Invalid cross-device link" }
```

Right now, we just silently ignore it and move on. With this diff, we'll
properly report that the write failed.

Reviewed By: farnz

Differential Revision: D26912990

fbshipit-source-id: 1b5a3c79c80f6b445e16e352e1c78aaec5e21850
2021-03-09 09:42:38 -08:00
svcscm
7e6a9da97e Updating submodules
Summary:
GitHub commits:

0e3c984d51
3af4c3db74
2806740f85
ec4278ce0f

Reviewed By: yns88

fbshipit-source-id: d8003d1453af77c1b78bb0de5857a8b75365b26d
2021-03-09 09:33:23 -08:00
Alex Hornby
a35bf5627d mononoke: force mysql index for changeset queries by repo_id and id
Summary: These queries are heavier than needed,  mysql optimizer needs a hint on correct index to use.

Reviewed By: johansglock

Differential Revision: D26911348

fbshipit-source-id: fc7e16edfb7d9f5e835b1e8bf7e38bab0730066e
2021-03-09 09:29:50 -08:00
Alex Hornby
84581f8a30 mononoke: add new bulkops benchmark for middle of repo bounds
Summary: Add a new benchmark so can measure timing in next diff.

Reviewed By: johansglock

Differential Revision: D26911760

fbshipit-source-id: 6f0df115478fa850e092d87e29d53708af8e7fa6
2021-03-09 09:29:50 -08:00
svcscm
0c74dd4b0a Updating submodules
Summary:
GitHub commits:

d00af38a77
deae128e45

Reviewed By: yns88

fbshipit-source-id: bf5bc7ff73da78febdd5b74ae44b0b0ac4b844ff
2021-03-09 09:29:49 -08:00
svcscm
b468baa7d6 Updating submodules
Summary:
GitHub commits:

b95b3e9251
10c600a309

Reviewed By: yns88

fbshipit-source-id: c20f781bf6e063b4ec0ee25f32d8d24cad549a88
2021-03-09 05:58:33 -08:00
Simon Farnsworth
9810ecbd1f Add slow poll watches to blobstore creation
Summary:
While rolling out Manifold C++ client, I noticed that blobrepo creation consistently complains about a 3 to 4 second poll time:

```
W0309 02:10:41.645635 3672554 [main] eden/mononoke/blobrepo/factory/src/lib.rs:161] Slow poll() ran for 3.170361901s
```
This future is `make_blobstore` from blobstore_factory

Add some slow poll watches to `make_blobstore` and the things it calls to help identify whatever is causing the task to hold the thread for 3 seconds - this often indicates somewhere that needs to use `blocking` to have a kernel sleep run in its own thread

Reviewed By: krallin

Differential Revision: D26910472

fbshipit-source-id: ed6d4cbb7fdf59ed1a243b6c0c0739dbf5502850
2021-03-09 05:52:21 -08:00
svcscm
b6bc67c67c Updating submodules
Summary:
GitHub commits:

2f7a126407

Reviewed By: yns88

fbshipit-source-id: ecf1f83264178ab56468ce12036e03b0e4097511
2021-03-09 05:02:19 -08:00
Stanislau Hlebik
00883f94d2 mononoke: remove unused getbundle code
Summary:
It was added for the initial rollout only so that we can fallback quickly if
needed (see D26221250 (7115cf31d2)). We can remove it now since the feature has been enabled
for a few weeks already with no big issues.

Reviewed By: krallin

Differential Revision: D26909490

fbshipit-source-id: 849fac4838c272e92a04a971869842156e88a1cf
2021-03-09 04:12:09 -08:00
Stanislau Hlebik
d01927844e mononoke: add additional getbundle logging
Summary:
I've been investigating getbundle on mononoke darkstorm, and it was hard to
understand what's going on. Adding more logs hopefully should be easier.

Also fix how we log `nodes_to_send` - previously `partial_result.partial`
wasn't counted. Now it should be fixed.

Reviewed By: krallin

Differential Revision: D26909296

fbshipit-source-id: 0af6f0b8d6af0350b5c87a20146ef8c7c64b3dc8
2021-03-09 03:20:10 -08:00
svcscm
4c26ea3547 Updating submodules
Summary:
GitHub commits:

7c7531bd24
42fd81f6a6

Reviewed By: yns88

fbshipit-source-id: 77e6587d8200419c3f8f6538acb08391e8b08694
2021-03-09 03:20:09 -08:00
svcscm
6c47afade8 Updating submodules
Summary:
GitHub commits:

1c382547db
33ff8c023a
ef392fb04e

Reviewed By: yns88

fbshipit-source-id: cde7a74169dee5b7016c16ee65a2765c4db0eb17
2021-03-08 18:08:47 -08:00
Katie Mancini
ea96dbbc2f make input loop eden top
Summary:
This teaches eden top to read input from the command line which we will use to
switch between help and main page as well as manipulate the process table.

Similar to the python version of eden top q exits. For now we spit back out all
of the other input we get.

Reviewed By: fanzeyi

Differential Revision: D26892622

fbshipit-source-id: 30039d2c2475c036c500c6cb1055c0b2d739691a
2021-03-08 18:03:57 -08:00
Katie Mancini
b3dffd4a54 Add template for rust eden top
Summary:
This adds a subcommand to the rust cli for top command. This sets us up to
start moving over the eden top functionality.

Reviewed By: fanzeyi

Differential Revision: D26892621

fbshipit-source-id: 1d0525291fbad1e5ea97ca22691eacb3fe42ffaf
2021-03-08 18:03:57 -08:00
svcscm
da064e43e7 Updating submodules
Summary:
GitHub commits:

4d235eee59

Reviewed By: yns88

fbshipit-source-id: 430b5abb7a0c0a52120ca0e561472dc320b39f81
2021-03-08 18:03:57 -08:00
svcscm
e194f7b2a1 Updating submodules
Summary:
GitHub commits:

0cbf5eb18e
41f4560c70
283fb09ac3

Reviewed By: yns88

fbshipit-source-id: 2d76e600fd4b32a6c356846900266d7ebe7d9dcc
2021-03-08 16:33:41 -08:00
Jun Wu
846050ea0f changelog: disable migrating backend for hgsql repos
Summary: hgsql can only use traditional revlog. Disable changelog migration for it.

Reviewed By: kulshrax

Differential Revision: D26891252

fbshipit-source-id: 36c5a448d4fcad15b3415e4534448a945f6d0b4b
2021-03-08 15:58:36 -08:00
Jun Wu
5b7b35d399 changelog: add missing changelog type logging
Summary: A few types weren't logged properly. Add them.

Reviewed By: kulshrax

Differential Revision: D26891253

fbshipit-source-id: 64f59c56663b4f395679307df4a75dcff1ff811f
2021-03-08 15:58:36 -08:00
Stefan Filip
95ab9679e2 error: fix constructor for NetworkError
Summary: hint is an expected valid keyword argument.

Reviewed By: quark-zju

Differential Revision: D26896943

fbshipit-source-id: 006413838fbfa63eccc24cbc98d0e713d5237a69
2021-03-08 14:56:26 -08:00
svcscm
db739ca8ee Updating submodules
Summary:
GitHub commits:

c20c89175f
e85a11d520
338523c0be
7b63354566
6c560e71c3
a6e6f2acb0

Reviewed By: yns88

fbshipit-source-id: 3ed716c8b75c161e542ad683b6ca3b2fda0b0090
2021-03-08 14:38:20 -08:00
Arun Kulshreshtha
c94dfbcc38 clidispatch: pass repo path to config contructor in from_raw_path
Summary: We weren't passing a repo path when initially loading the repo's config in `clidispatch`. This meant that the resulting `ConfigSet` didn't contain values from the shared-repo `.hgrc.dynamic`. Evidently, some other code path in Python would eventually add these values, but this meant that pure-Rust commands could not see config values set via dynamicconfig. Passing the path fixes the problem.

Reviewed By: DurhamG

Differential Revision: D26508980

fbshipit-source-id: 65f187d18098a08c81325e78cb02a8ed854c739a
2021-03-08 13:31:10 -08:00
svcscm
bbb0f7af67 Updating submodules
Summary:
GitHub commits:

2a16bc38d3
12e3152b04
394c09d937
1b94301891
177b5de4fc

Reviewed By: yns88

fbshipit-source-id: 69c45ca59e36ab2e78c3ee2c5a85f33e56c6e4af
2021-03-08 13:31:10 -08:00
svcscm
664397d785 Updating submodules
Summary:
GitHub commits:

4766723339
25dc8c1c46
aa4ae13e9c
a45567ffe9
5726f51e2d
8310900d55

Reviewed By: yns88

fbshipit-source-id: 0c64b295c9e4db3c4ad2a1de37a21e0dc09224db
2021-03-08 12:46:56 -08:00
Ran Yaniv
e5d163411a run autocargo
Summary: Run autocargo.

Reviewed By: c-ryan747

Differential Revision: D26878408

fbshipit-source-id: 14e2efb280c906fa7a79253da74d7518e1a89079
2021-03-08 12:40:50 -08:00
Jun Wu
02ff6e200c remotenames: clean up bookmarks that are ancestors of master
Summary:
See the previous diff for motivation. This removes bookmarks that are
ancestor of master, too. This is important in practice.

Reviewed By: DurhamG

Differential Revision: D26889412

fbshipit-source-id: 255722ed5b486e88ef56e7e378fae3f1113d5fbe
2021-03-08 11:28:40 -08:00
Jun Wu
2e353eb4ce tests: add a test showing release bookmarks aren't automatically cleaned up
Summary:
The auto cleanup was conservative. It keeps `::draft()`. But that means
ancestors of public commits are not cleaned up. Not all release branches
branch off the master branch.

Reviewed By: DurhamG

Differential Revision: D26889413

fbshipit-source-id: c6a8e3f32cf1f7d2ffe74b7ecd183f4e583949bb
2021-03-08 11:28:40 -08:00
svcscm
de70ded4b3 Updating submodules
Summary:
GitHub commits:

8f50aff1d0

Reviewed By: yns88

fbshipit-source-id: 8e869dc50ebdcf1c243ef5608683ea8949054745
2021-03-08 11:28:40 -08:00
svcscm
692d2522af Updating submodules
Summary:
GitHub commits:

9a5fc814b3
0e3a56d886
25d16203c8

Reviewed By: yns88

fbshipit-source-id: 2b77f4c7fb44f025826f922763c6d4f64731ebe2
2021-03-08 10:26:47 -08:00
Stanislau Hlebik
cc0c8cd0ff mononoke: allow changing xrepo mapping extra for if config is set
Summary:
Normally we prohibit landing commits that might accidentally
change the x-repo mapping. However we do want to allow landing commits like
that to backup repos, because backup repos should have all commits as their
counterpart repositories.

This change also has another side-effect - we don't call `load_additional_changeset()`
which can be very expensive for backup repos because of the issues in configuration -
in particular, we don't have `hooks_ancestors_of` option set, and that caused all ancestors to be considered
as "additional changesets". It would make since more properly later.

Differential Revision: D26883910

fbshipit-source-id: 07ceb7b96bc6cae851ac6ff57071eff5cef387e4
2021-03-08 08:39:30 -08:00
Stefan Filip
f172c2f8ae mononokepeer: wrap read exceptions with NetworkError
Summary:
This allows for errors raised in these cases to be retried. Most notable is
the timeout error.

Reviewed By: johansglock

Differential Revision: D26855441

fbshipit-source-id: 6137ed1755072d43dbdd25fa092ddb21c8669aa3
2021-03-08 08:06:01 -08:00
Stefan Filip
573c25553a mononokepeer: add timeout to socket
Summary:
No timeout is set up by default so the process wait forever when reading bytes
in cases where the connection is lost somehow.

Reviewed By: johansglock

Differential Revision: D26855443

fbshipit-source-id: d741f73e7186fe862f3d78a806f3219c2cbe7e0a
2021-03-08 08:06:00 -08:00
Stefan Filip
882c8c70a7 mononokepeer: prefer NetworkError over Abort
Summary:
Abort one of the most general exceptions in Mercurial. In theory it should be
something that isn't handled. We can say at least that it shouldn't be retried.
For thing that may be transient it is better to use a different type of
exception. NetworkError is something is checked and retries in a few places so
that seems like a natural candidate.

Reviewed By: johansglock

Differential Revision: D26855444

fbshipit-source-id: f15c723293a416b5f44a6592927e3500f3b0b7d5
2021-03-08 08:06:00 -08:00
Stefan Filip
51367f0312 error: generalize NetworkError to all network read errors
Summary: Timeouts are another class of errors that are relevant.

Reviewed By: johansglock

Differential Revision: D26855442

fbshipit-source-id: 8ebb83714fa3d7a2f4efcbed8bd512c98301b49d
2021-03-08 08:06:00 -08:00
svcscm
dc63c9b353 Updating submodules
Summary:
GitHub commits:

122ce9f76b
692604cf54

Reviewed By: wittgenst

fbshipit-source-id: b69a42a55292def4688297e36c4ebbb43f6425d7
2021-03-08 08:06:00 -08:00