Commit Graph

65069 Commits

Author SHA1 Message Date
svcscm svcscm
b1fa8a07d6 Updating submodules
Summary:
GitHub commits:

5f6d41f9d0
52ecb153bb
702bca1889
3b8895adb6
2caeb94ed4

Reviewed By: jurajh-fb

fbshipit-source-id: be5b02f4dd60ba66d29daad38b72f40fdec03958
2021-04-30 05:45:38 -07:00
Alex Hornby
d7330b7557 mononoke: walk from BonsaiHgMapping to HgChangesetViaBonsai for consistency
Summary:
Previously there were two different paths to HgChangeset.  This diff unifies them, so that when walker state.rs is checking for a previous visit it will find that it happened.

For existing walks of changesets in the NewestFirst direction this wasn't causing a problem, however the next diff in stack adds support for OldestFirst walks.  In the OldestFirst case the mismatch in paths to HgChangeset was leaving a deferred edge to visit when everything should have been visited in previous chunks.

Differential Revision: D28095569

fbshipit-source-id: ccba4a679fc28bde042cfc222e5097c84fa968c0
2021-04-30 05:43:39 -07:00
svcscm svcscm
a72f378ffb Updating submodules
Summary:
GitHub commits:

118a054ddc
87eff397f3
3b6ce09273
62d877b31e
16869fd1f7

Reviewed By: jurajh-fb

fbshipit-source-id: 78e334a96b9df2072eb550cadcba82f00b8c1573
2021-04-30 05:09:19 -07:00
svcscm svcscm
6ee7919858 Updating submodules
Summary:
GitHub commits:

728e05da07
e944755038
ac0af5746d
4ae579b7b3
7e75582a0a
c666c495a5

Reviewed By: jurajh-fb

fbshipit-source-id: 53df9f424178eb4ba0915ce7b84635ec98daaa19
2021-04-30 04:25:31 -07:00
svcscm svcscm
303da903d3 Updating submodules
Summary:
GitHub commits:

d9f4c283ec
b594ad4a39
5805f84b78
91a85f67e1
b4b69ba85e
dd7649fe2a
e46ed2a765
4685015e85
f877c525dc

Reviewed By: jurajh-fb

fbshipit-source-id: d5387306c9e73b13d36e47b7b0323f5dc394085b
2021-04-30 03:48:59 -07:00
Thomas Orozco
df72cc772f mononoke: enable level filtering when printing information about stdlog
Summary:
Right now we write straight to a logger with no filter, so no matter the log
level we print this stuff out. Let's fix it.

While we're at it, move this back to debug level.

I'd made this trace in my recent cmdlib refactoring (which resulted in us
properly initializing logging in all binaries), since I assumed we just had level
filtering working but with debug-logging enabled and I didn't want to have to
update every single test, but it turns out that the reason we didn't print it
out at trace is just because thats not enabled at all in our slog build:
D28097080.

Reviewed By: StanislavGlebik

Differential Revision: D28116053

fbshipit-source-id: f59d9a70ea3c3d834adea16f2686bfc244672b14
2021-04-30 03:24:56 -07:00
svcscm svcscm
dcc9950222 Updating submodules
Summary:
GitHub commits:

97c0de3135
f0adc031b8
7ab23b651b
6a58519a73
f0258e9b96
40d4dfe30b
a6f93a6bb1
b76c964432
84f934c721
761a8c094d
fb672cf305

Reviewed By: jurajh-fb

fbshipit-source-id: e04ed3befa06a1df2b00bc7a38713c58a04223d7
2021-04-30 03:03:13 -07:00
Simon Farnsworth
3f33c7ac4c Account for zstd non-determinism in packer test
Summary: The precise compressed size of big blobs in zstd varies between runs. Glob out the exact size

Reviewed By: StanislavGlebik

Differential Revision: D28116066

fbshipit-source-id: 990add820de6c8cb0029805bc1de304fdf83acba
2021-04-30 02:46:42 -07:00
svcscm svcscm
ce13f763a9 Updating submodules
Summary:
GitHub commits:

14077e9cf2
b0fccaf86f
c1158351fe
9552fbcd65
ad6c0db847
9af2090bbe
9593e16ed1
a1a6da0071
4d07cb7c28
23adf8ebab
03b4f5b0b5

Reviewed By: jurajh-fb

fbshipit-source-id: 2c865ae047c7313c26f0df3510f88c7e63a09cc4
2021-04-30 02:34:12 -07:00
Stanislau Hlebik
45c3ba7d02 mononoke: add fastlog to warm bookmark cache
Summary:
It wasn't in warm bookmark cache, but it was an oversight - there's no reason
for it to not be here. Let's add it, since in case of crashlooping derived data
tailer (see attached task T89911396) there might be nothing to derive fastlog
data structure, and we end up with a long queue to derive.

Reviewed By: krallin

Differential Revision: D28114533

fbshipit-source-id: feb29c07d90be6250c5385ae9f2fb13eb52eedba
2021-04-30 01:50:25 -07:00
svcscm svcscm
cd38e3f12b Updating submodules
Summary:
GitHub commits:

a78cb3aa52
4d3fd7072e
3bab334d21
e08e3eb5b0
bd0ef3bca1
5ce0eed074

Reviewed By: jurajh-fb

fbshipit-source-id: fe4786c8bcd8070bb339bb91c77bd2ec6edf8c06
2021-04-30 00:34:41 -07:00
svcscm svcscm
00e990098b Updating submodules
Summary:
GitHub commits:

6bf526e107
e8507927c7
561d30acc4
c898856f2a
e6edae94f6

Reviewed By: jurajh-fb

fbshipit-source-id: 31fb66765c816325110296d6c4a98674ab9fbb60
2021-04-29 23:12:17 -07:00
svcscm svcscm
0a691d7dce Updating submodules
Summary:
GitHub commits:

c5ef22ac83
6c0d8601c6
58c7f2b0db
71430335c6
a21d11c509
2baf8d6e57

Reviewed By: jurajh-fb

fbshipit-source-id: a9a888f3a3973d74478c664bcba1feff2a12a8db
2021-04-29 22:31:43 -07:00
svcscm svcscm
a016afed8c Updating submodules
Summary:
GitHub commits:

3e83793e76
23ed58d5e6
4837b35d8a
de4bfa0fc7
1ccc377ee9
e3f7316a17
aeff1d453d
6d4d603558
4fea09631b

Reviewed By: jurajh-fb

fbshipit-source-id: ef4c43786f0d76ccd14521abc7955bd30b56edce
2021-04-29 21:53:41 -07:00
svcscm svcscm
5f8f44ce43 Updating submodules
Summary:
GitHub commits:

b87655effc
72d7c305dc
8fa0f4a15e
5af505a2ec
b4bf323730
4795d5ac6e
87b7f999fa
b286a2648d
8ab21fd268
f0285904d5
389f94ef5b

Reviewed By: jurajh-fb

fbshipit-source-id: a4800eb6b817d115e21489a96a2f3fc39542ce94
2021-04-29 19:46:21 -07:00
Xavier Deguillard
fc382774d6 service: remove dependency on curl
Summary:
From what I can see, this was added when EdenFS had a Mononoke store, which is
now long gone, thus we should be able to remove the Curl dependency altogether.

Reviewed By: fanzeyi

Differential Revision: D28037816

fbshipit-source-id: 834f7db64bab5dda1748ad2f033c27a2854b0ba4
2021-04-29 19:41:04 -07:00
svcscm svcscm
29e15d0253 Updating submodules
Summary:
GitHub commits:

1ff7dfa961
1bbc6025c3
ad9ad2d92f
3cdd300904
62cf07fcf8
96dd76ea39
13893a6d01
530842f183
17e7ff366a
c8962a8f8e

Reviewed By: jurajh-fb

fbshipit-source-id: 6f395e4ba0b54bc477d6f04b26ccaf8f82c930e6
2021-04-29 19:04:21 -07:00
svcscm svcscm
c730117a64 Updating submodules
Summary:
GitHub commits:

6f53685ac3
4a1c9128ae
fc26a824d3
e30073f767
84741b4b4c
345541a6ed
66868da88c
b551670fa7
0d89985826
58b20966ed
7af6eab3bd

Reviewed By: jurajh-fb

fbshipit-source-id: fa2e2adec28696bbeb0a1d851397b698367018e3
2021-04-29 18:37:26 -07:00
svcscm svcscm
a75d47aa4e Updating submodules
Summary:
GitHub commits:

147b702c48
67e8a5b22e
faf1c11d96
903990dbc2
269c7c31ce
c0de151bee
71ff16a9e8

Reviewed By: jurajh-fb

fbshipit-source-id: 6b47479f1082427df1b0598ff8b824d622758dae
2021-04-29 17:34:54 -07:00
Xavier Deguillard
ddf6c2dc5c fuse: remove @manual from FuseTypes.h
Summary: Looks like these aren't needed since these files are owned by a TARGETS file.

Reviewed By: genevievehelsel

Differential Revision: D28101197

fbshipit-source-id: d790530227641bf25e48bd96c8a95dd31f08a954
2021-04-29 17:30:16 -07:00
svcscm svcscm
02580b6c8e Updating submodules
Summary:
GitHub commits:

08e8ed98a1

Reviewed By: jurajh-fb

fbshipit-source-id: aed0c175d754c7c43c879ee24ef39458a665d5da
2021-04-29 16:21:52 -07:00
Xavier Deguillard
5a6172a8e6 autodeps: remove @manual for cpptoml
Summary:
Now that autodeps knows where to find cpptoml.h, we no longer need these
manual annotation.

Reviewed By: kmancini

Differential Revision: D28100956

fbshipit-source-id: 463b73834c500c1d16a4a769af3655938124d49d
2021-04-29 16:19:09 -07:00
Jeremy Fitzhardinge
f4f52c1d96 mononoke/mercurial_types: remove unneeded vec![] temporaries
Summary:
For no particular reason I was looking at this and saw a bunch of
unneeded `vec![]` temporaries which could be replaced with arrays or slices.

Reviewed By: krallin

Differential Revision: D28073693

fbshipit-source-id: 7fca3b4c7b40cc380b4b128e9809912b7b9ba1f7
2021-04-29 15:49:58 -07:00
svcscm svcscm
7dea42e313 Updating submodules
Summary:
GitHub commits:

9903f4099b
c13c57f5f7
783375b230
e9a2124f46

Reviewed By: jurajh-fb

fbshipit-source-id: 7580516fb5aca93c441f2e768054f4b5f0fcea48
2021-04-29 15:46:11 -07:00
svcscm svcscm
9ccdd077d2 Updating submodules
Summary:
GitHub commits:

7e62468db4
2189ed489f
0fabe16ba2
3c02bc7907
b58a9f3a3f
ae20ff8cf4

Reviewed By: jurajh-fb

fbshipit-source-id: 7f1ff757e005c4125ab5b72c14db8298cc881228
2021-04-29 15:19:19 -07:00
Mateusz Kwapich
43c2f9f88e error out when fetching NULL data from hgsql
Summary:
The original bug that resulted in empty revisions being pulled is long-fixed:
T28553115. I'm planning to make data1 nullable so I can reclaim space by removing older
revs.

Reviewed By: DurhamG

Differential Revision: D28096278

fbshipit-source-id: a57da458df115dcbdf544e2151aa327651190c1a
2021-04-29 14:45:12 -07:00
Mateusz Kwapich
d1064681ee bring back debugephemeralcommit.py
Summary: andll removed it (probably by accident) in D27722921 (80adbe385c)

Reviewed By: andll

Differential Revision: D28096279

fbshipit-source-id: 0d3e9aee4c22803680cee8d5e3a40d51d7f36b7b
2021-04-29 14:45:12 -07:00
Mateusz Kwapich
2a23089e9a hgsql: fix tests
Summary:
This enlists hgsql tests to the lists of tests using revision numbers and
marks some racy lines as optional

Reviewed By: quark-zju

Differential Revision: D28096282

fbshipit-source-id: eb8406cb74f3338d13d4109fce35f969ff9e3b79
2021-04-29 14:45:12 -07:00
Mateusz Kwapich
7a6c3e090f lib: remove unused C code
Summary:
This is a hg-sever backport of fix from D27659634 (8e8aaa61d6)

Those are not used. Recently we saw build issues like:

  lib/third-party/sha1dc/sha1.c:8:10: fatal error: string.h: No such file or directory
   #include <string.h>
            ^~~~~~~~~~

Possibly by some compiler flags disabling stdlib. Since we don't need
the C code let's just remove them.

Reviewed By: StanislavGlebik

Differential Revision: D28096283

fbshipit-source-id: 6c5390d26264e1e39f99b29dec8608d92e5ae572
2021-04-29 14:45:12 -07:00
svcscm svcscm
2d13700114 Updating submodules
Summary:
GitHub commits:

24c4d606d3
b107cb54e4
fbb6d8a15f
bc68eb7b0f
00e948e5a1
c63953cc8b

Reviewed By: jurajh-fb

fbshipit-source-id: 5871389e99f400619abe989013f1098630c3f3ed
2021-04-29 14:03:39 -07:00
Thomas Orozco
a0debf738b mononoke/lfs_server: log client attempts left
Summary: - Like it says in the title.

Reviewed By: HarveyHunt

Differential Revision: D28092796

fbshipit-source-id: 01816f815148aca6c86078fb7dec616ecf53095c
2021-04-29 13:54:07 -07:00
Thomas Orozco
bef2578fc2 revisionstore/lfs: treat backoffs separately from transfer errors
Summary:
This updates hg to have a different amount of retry for backoffs requested by
the server and errors.

The rationale is that backoffs are fairly well understood and usually caused by
a surge in traffic where everybody wants the same data (in which case we should
be willing to wait to get it because there is literally no alternative),
whereas general errors aren't predictable in the same way.

We're now effectively at a point on the server side where _all_ our instances
have the exact same load, so if any server is telling you to backoff, that
pretty much guarantees that the whole tier has too much traffic to deal with.

This leaves us with two options:

- Tell clients to wait longer and smooth out the traffic surge.
- Add enough capacity that even our biggest surges don't result in _any_
  throttling.

The latter is a bit unrealistic unrealistic given we routinely get egress
variations in excess of 5x (here's an example: https://fburl.com/ods/pidsrqnl),
so this does the former.

This also updates the client to tell the server how many attempts it has left
in addition to how many it used up so far. How many are left is more meaningful
for alerting!

Finally, it adds a bit of logging so that in debug mode you can see this
happening.

Reviewed By: quark-zju

Differential Revision: D28092797

fbshipit-source-id: f61410e39c4a3e3356371a3c7bd7892de4beacc8
2021-04-29 13:54:07 -07:00
Jun Wu
d3de89f62c changelog: do not consider integers revs for shortest
Summary:
After D27144492 (48cd15ab14) we disabled revision number resolution. There is no need to
consider it when calculating shortest prefix.

Reviewed By: DurhamG

Differential Revision: D28072997

fbshipit-source-id: 832017c7b626265eb8cd2dd78946a03c4e7228f6
2021-04-29 13:39:31 -07:00
Zeyi (Rice) Fan
e94d69dcd5 utils: define symlink type for Windows
Summary:
This diff defines symlink type in `DirType`.

Even though it is not directly used in the FSCK diff. This will allow us to support symlink in EdenFS Windows in the future.

Reviewed By: genevievehelsel

Differential Revision: D28016305

fbshipit-source-id: 67c1aa22e39198f9c91845129695f27b8303a5f1
2021-04-29 13:17:54 -07:00
svcscm svcscm
35b4530028 Updating submodules
Summary:
GitHub commits:

ac5dc594f8
e56a043ac6
76f53ef078
93592cc349
e5e2d4630e
5d8daf69f0
09e62bf126
67c8e82ed1
1a21bee17f

Reviewed By: jurajh-fb

fbshipit-source-id: a4590a5e347efc3f4da50645f2a41175f3e766a3
2021-04-29 12:53:18 -07:00
svcscm svcscm
f4c97470f5 Updating submodules
Summary:
GitHub commits:

3e057d9e60
351b579301
a14ad4bc30
56855e84d7
20cab44f64
5ec0d38bf6
dfb3dcee18
d02dfae538
8d2eddb9c5
ef0f1c4da9
d8cd7f09bb
f60aa9d195
6d7ecf4d8b

Reviewed By: jurajh-fb

fbshipit-source-id: 65663184c36fbba3082544b1f608c858798c9c20
2021-04-29 12:21:52 -07:00
svcscm svcscm
b9bb7b4569 Updating submodules
Summary:
GitHub commits:

287a315f53
c958a0cacb
985cfd73d4
cdf0badc83
bee13ee466
104b00f0fe
caaab5f665
fd554e17ff

Reviewed By: jurajh-fb

fbshipit-source-id: 15b8c841256ca14371d3c5501b5c43f8011a4e8a
2021-04-29 11:12:08 -07:00
Alex Hornby
c89d21a9d6 mononoke: add strum to bulkops Direction enum
Summary: Add strum derivations to bulkops so we can use them in command line parsing later in stack.

Differential Revision: D28069912

fbshipit-source-id: 4d997e20e18f2011b51933ed4322c85bb7468980
2021-04-29 11:03:47 -07:00
Xavier Deguillard
3868dd01fc inodes: File::fallocate should return a Future
Summary:
We were ignoring the return value of runWhileMaterialized, and thus we were
returning to FUSE before fallocate returned.

Reviewed By: fanzeyi

Differential Revision: D28081991

fbshipit-source-id: f398942ddb2432e48e80c148abc8edb7e5ada71d
2021-04-29 09:51:48 -07:00
Alex Hornby
aeddca942b mononoke: connect up mtime tracking to walker scrub
Summary: Start logging mtime as relatedness key in the walker scrub pack info output

Differential Revision: D28055637

fbshipit-source-id: 4c24c5f2af0414ae7df17ade69bba9ff18861264
2021-04-29 09:32:47 -07:00
Alex Hornby
244b4bdd4f mononoke: connect up path hash tracking to walker scrub
Summary: Start logging path hashes in the walker scrub pack info output

Differential Revision: D28031871

fbshipit-source-id: d610aeb1410b58611147b73eb47e123f997d4cf3
2021-04-29 09:32:47 -07:00
Thomas Orozco
9c7aa6aaf7 third-party/rust: remove patches for Tokio 0.2 & Hyper 0.2
Summary:
We used to carry patches for Tokio 0.2 to add support for disabling Tokio coop
(which was necessary to make Mononoke work with it), but this was upstreamed
in Tokio 1.x (as a different implementation), so that's no longer needed. Nobody
else besides Mononoke was using this.

For Hyper we used to carry a patch with a bugfix. This was also fixed in Tokio
1.x-compatible versions of Hyper. There are still users of hyper-02 in fbcode.
However, this is only used for servers and only when accepting websocket
connections, and those users are just using Hyper as a HTTP client.

Reviewed By: farnz

Differential Revision: D28091331

fbshipit-source-id: de13b2452b654be6f3fa829404385e80a85c4420
2021-04-29 08:07:45 -07:00
Thomas Orozco
ffed22260d third-party/rust: remove Gotham 0.2
Summary:
This used to be used by Mononoke, but we're now on Tokio 1.x and on
corresponding versions of Gotham so it's not needed anymore.

Reviewed By: farnz

Differential Revision: D28091091

fbshipit-source-id: a58bcb4ba52f3f5d2eeb77b68ee4055d80fbfce2
2021-04-29 08:07:45 -07:00
svcscm svcscm
46354e4c17 Updating submodules
Summary:
GitHub commits:

d85ae094eb
8f236c95f1
bb8f85bf21
60c8005870
e05c431f55
85becd94c1
00ff6f4709
e6019b317f

Reviewed By: jurajh-fb

fbshipit-source-id: 7f9907ce23ab5ea6710a8e2cc4e2989e56b45538
2021-04-29 07:49:15 -07:00
Alex Hornby
40a7e12e56 mononoke: make scrub EmptyRoute produce WalkKeyOptPathHash
Summary:
Connect up the scrub stream types so they will be uniform for scrubs that log pack info and those that do not.

This is in preprepation for the next diff which connects up the pack info logging of path hashes to scrub.    CI for this diff verifies its not broken the non-path tracking case.

Differential Revision: D28031868

fbshipit-source-id: 7bf91eb1778f57487f6a2847f215cf7f5cd2dff7
2021-04-29 07:46:29 -07:00
Alex Hornby
58f52f6422 mononoke: make WalkKeyOptPath generic over path and pathhash
Summary: This moves evolve_path up to WrappedPathLike so that we can use sample route evolution logic for routes that track paths (e.g. corpus sampling) and path hashes (e.g. scrub, where path hashes take less memory than full paths).

Differential Revision: D28031867

fbshipit-source-id: cdabdc466158a8db1c770536747c996dddb27e71
2021-04-29 07:46:29 -07:00
Alex Hornby
b296a73c2c mononoke: name the fields in WalkPayloadMtime
Summary: Name the fields rather than leave it as a tuple struct.  This makes it a bit easier to work with in the rest of the stack

Differential Revision: D28062254

fbshipit-source-id: 9e5202b4d6f1f29d44d98b86aa9b6ddb97d821eb
2021-04-29 07:46:29 -07:00
Alex Hornby
ff462f5422 mononoke: move filter_repo_path to be a method on NodeType
Summary: Makes more sense for this to be a method on NodeType

Differential Revision: D28031869

fbshipit-source-id: 1ddbafa0d7634ac67fd8d5112e6f57759ed91638
2021-04-29 07:46:29 -07:00
Alex Hornby
4e7f0888bf mononoke: name the fields in WalkKeyOptPath
Summary: Name the fields rather than leave it as a tuple struct

Differential Revision: D28031866

fbshipit-source-id: 039f004e0b81294aa6d6b13e79cb45ee2b84567c
2021-04-29 07:46:29 -07:00
Alex Hornby
901fe40f5f mononoke: introduce WrappedPathLike to walker
Summary: This new trait abstracts across WrappedPath and WrapperPathHash.  Later in the stack I make path tracking use this to track either full paths (for corpus sampling) or path hashes (for logging from scrub).

Differential Revision: D28031870

fbshipit-source-id: d1c57230f68fffff179929a3cb92c82d92e0588c
2021-04-29 07:46:29 -07:00