Commit Graph

65201 Commits

Author SHA1 Message Date
svcscm svcscm
a75d47aa4e Updating submodules
Summary:
GitHub commits:

147b702c48
67e8a5b22e
faf1c11d96
903990dbc2
269c7c31ce
c0de151bee
71ff16a9e8

Reviewed By: jurajh-fb

fbshipit-source-id: 6b47479f1082427df1b0598ff8b824d622758dae
2021-04-29 17:34:54 -07:00
Xavier Deguillard
ddf6c2dc5c fuse: remove @manual from FuseTypes.h
Summary: Looks like these aren't needed since these files are owned by a TARGETS file.

Reviewed By: genevievehelsel

Differential Revision: D28101197

fbshipit-source-id: d790530227641bf25e48bd96c8a95dd31f08a954
2021-04-29 17:30:16 -07:00
svcscm svcscm
02580b6c8e Updating submodules
Summary:
GitHub commits:

08e8ed98a1

Reviewed By: jurajh-fb

fbshipit-source-id: aed0c175d754c7c43c879ee24ef39458a665d5da
2021-04-29 16:21:52 -07:00
Xavier Deguillard
5a6172a8e6 autodeps: remove @manual for cpptoml
Summary:
Now that autodeps knows where to find cpptoml.h, we no longer need these
manual annotation.

Reviewed By: kmancini

Differential Revision: D28100956

fbshipit-source-id: 463b73834c500c1d16a4a769af3655938124d49d
2021-04-29 16:19:09 -07:00
Jeremy Fitzhardinge
f4f52c1d96 mononoke/mercurial_types: remove unneeded vec![] temporaries
Summary:
For no particular reason I was looking at this and saw a bunch of
unneeded `vec![]` temporaries which could be replaced with arrays or slices.

Reviewed By: krallin

Differential Revision: D28073693

fbshipit-source-id: 7fca3b4c7b40cc380b4b128e9809912b7b9ba1f7
2021-04-29 15:49:58 -07:00
svcscm svcscm
7dea42e313 Updating submodules
Summary:
GitHub commits:

9903f4099b
c13c57f5f7
783375b230
e9a2124f46

Reviewed By: jurajh-fb

fbshipit-source-id: 7580516fb5aca93c441f2e768054f4b5f0fcea48
2021-04-29 15:46:11 -07:00
svcscm svcscm
9ccdd077d2 Updating submodules
Summary:
GitHub commits:

7e62468db4
2189ed489f
0fabe16ba2
3c02bc7907
b58a9f3a3f
ae20ff8cf4

Reviewed By: jurajh-fb

fbshipit-source-id: 7f1ff757e005c4125ab5b72c14db8298cc881228
2021-04-29 15:19:19 -07:00
Mateusz Kwapich
43c2f9f88e error out when fetching NULL data from hgsql
Summary:
The original bug that resulted in empty revisions being pulled is long-fixed:
T28553115. I'm planning to make data1 nullable so I can reclaim space by removing older
revs.

Reviewed By: DurhamG

Differential Revision: D28096278

fbshipit-source-id: a57da458df115dcbdf544e2151aa327651190c1a
2021-04-29 14:45:12 -07:00
Mateusz Kwapich
d1064681ee bring back debugephemeralcommit.py
Summary: andll removed it (probably by accident) in D27722921 (80adbe385c)

Reviewed By: andll

Differential Revision: D28096279

fbshipit-source-id: 0d3e9aee4c22803680cee8d5e3a40d51d7f36b7b
2021-04-29 14:45:12 -07:00
Mateusz Kwapich
2a23089e9a hgsql: fix tests
Summary:
This enlists hgsql tests to the lists of tests using revision numbers and
marks some racy lines as optional

Reviewed By: quark-zju

Differential Revision: D28096282

fbshipit-source-id: eb8406cb74f3338d13d4109fce35f969ff9e3b79
2021-04-29 14:45:12 -07:00
Mateusz Kwapich
7a6c3e090f lib: remove unused C code
Summary:
This is a hg-sever backport of fix from D27659634 (8e8aaa61d6)

Those are not used. Recently we saw build issues like:

  lib/third-party/sha1dc/sha1.c:8:10: fatal error: string.h: No such file or directory
   #include <string.h>
            ^~~~~~~~~~

Possibly by some compiler flags disabling stdlib. Since we don't need
the C code let's just remove them.

Reviewed By: StanislavGlebik

Differential Revision: D28096283

fbshipit-source-id: 6c5390d26264e1e39f99b29dec8608d92e5ae572
2021-04-29 14:45:12 -07:00
svcscm svcscm
2d13700114 Updating submodules
Summary:
GitHub commits:

24c4d606d3
b107cb54e4
fbb6d8a15f
bc68eb7b0f
00e948e5a1
c63953cc8b

Reviewed By: jurajh-fb

fbshipit-source-id: 5871389e99f400619abe989013f1098630c3f3ed
2021-04-29 14:03:39 -07:00
Thomas Orozco
a0debf738b mononoke/lfs_server: log client attempts left
Summary: - Like it says in the title.

Reviewed By: HarveyHunt

Differential Revision: D28092796

fbshipit-source-id: 01816f815148aca6c86078fb7dec616ecf53095c
2021-04-29 13:54:07 -07:00
Thomas Orozco
bef2578fc2 revisionstore/lfs: treat backoffs separately from transfer errors
Summary:
This updates hg to have a different amount of retry for backoffs requested by
the server and errors.

The rationale is that backoffs are fairly well understood and usually caused by
a surge in traffic where everybody wants the same data (in which case we should
be willing to wait to get it because there is literally no alternative),
whereas general errors aren't predictable in the same way.

We're now effectively at a point on the server side where _all_ our instances
have the exact same load, so if any server is telling you to backoff, that
pretty much guarantees that the whole tier has too much traffic to deal with.

This leaves us with two options:

- Tell clients to wait longer and smooth out the traffic surge.
- Add enough capacity that even our biggest surges don't result in _any_
  throttling.

The latter is a bit unrealistic unrealistic given we routinely get egress
variations in excess of 5x (here's an example: https://fburl.com/ods/pidsrqnl),
so this does the former.

This also updates the client to tell the server how many attempts it has left
in addition to how many it used up so far. How many are left is more meaningful
for alerting!

Finally, it adds a bit of logging so that in debug mode you can see this
happening.

Reviewed By: quark-zju

Differential Revision: D28092797

fbshipit-source-id: f61410e39c4a3e3356371a3c7bd7892de4beacc8
2021-04-29 13:54:07 -07:00
Jun Wu
d3de89f62c changelog: do not consider integers revs for shortest
Summary:
After D27144492 (48cd15ab14) we disabled revision number resolution. There is no need to
consider it when calculating shortest prefix.

Reviewed By: DurhamG

Differential Revision: D28072997

fbshipit-source-id: 832017c7b626265eb8cd2dd78946a03c4e7228f6
2021-04-29 13:39:31 -07:00
Zeyi (Rice) Fan
e94d69dcd5 utils: define symlink type for Windows
Summary:
This diff defines symlink type in `DirType`.

Even though it is not directly used in the FSCK diff. This will allow us to support symlink in EdenFS Windows in the future.

Reviewed By: genevievehelsel

Differential Revision: D28016305

fbshipit-source-id: 67c1aa22e39198f9c91845129695f27b8303a5f1
2021-04-29 13:17:54 -07:00
svcscm svcscm
35b4530028 Updating submodules
Summary:
GitHub commits:

ac5dc594f8
e56a043ac6
76f53ef078
93592cc349
e5e2d4630e
5d8daf69f0
09e62bf126
67c8e82ed1
1a21bee17f

Reviewed By: jurajh-fb

fbshipit-source-id: a4590a5e347efc3f4da50645f2a41175f3e766a3
2021-04-29 12:53:18 -07:00
svcscm svcscm
f4c97470f5 Updating submodules
Summary:
GitHub commits:

3e057d9e60
351b579301
a14ad4bc30
56855e84d7
20cab44f64
5ec0d38bf6
dfb3dcee18
d02dfae538
8d2eddb9c5
ef0f1c4da9
d8cd7f09bb
f60aa9d195
6d7ecf4d8b

Reviewed By: jurajh-fb

fbshipit-source-id: 65663184c36fbba3082544b1f608c858798c9c20
2021-04-29 12:21:52 -07:00
svcscm svcscm
b9bb7b4569 Updating submodules
Summary:
GitHub commits:

287a315f53
c958a0cacb
985cfd73d4
cdf0badc83
bee13ee466
104b00f0fe
caaab5f665
fd554e17ff

Reviewed By: jurajh-fb

fbshipit-source-id: 15b8c841256ca14371d3c5501b5c43f8011a4e8a
2021-04-29 11:12:08 -07:00
Alex Hornby
c89d21a9d6 mononoke: add strum to bulkops Direction enum
Summary: Add strum derivations to bulkops so we can use them in command line parsing later in stack.

Differential Revision: D28069912

fbshipit-source-id: 4d997e20e18f2011b51933ed4322c85bb7468980
2021-04-29 11:03:47 -07:00
Xavier Deguillard
3868dd01fc inodes: File::fallocate should return a Future
Summary:
We were ignoring the return value of runWhileMaterialized, and thus we were
returning to FUSE before fallocate returned.

Reviewed By: fanzeyi

Differential Revision: D28081991

fbshipit-source-id: f398942ddb2432e48e80c148abc8edb7e5ada71d
2021-04-29 09:51:48 -07:00
Alex Hornby
aeddca942b mononoke: connect up mtime tracking to walker scrub
Summary: Start logging mtime as relatedness key in the walker scrub pack info output

Differential Revision: D28055637

fbshipit-source-id: 4c24c5f2af0414ae7df17ade69bba9ff18861264
2021-04-29 09:32:47 -07:00
Alex Hornby
244b4bdd4f mononoke: connect up path hash tracking to walker scrub
Summary: Start logging path hashes in the walker scrub pack info output

Differential Revision: D28031871

fbshipit-source-id: d610aeb1410b58611147b73eb47e123f997d4cf3
2021-04-29 09:32:47 -07:00
Thomas Orozco
9c7aa6aaf7 third-party/rust: remove patches for Tokio 0.2 & Hyper 0.2
Summary:
We used to carry patches for Tokio 0.2 to add support for disabling Tokio coop
(which was necessary to make Mononoke work with it), but this was upstreamed
in Tokio 1.x (as a different implementation), so that's no longer needed. Nobody
else besides Mononoke was using this.

For Hyper we used to carry a patch with a bugfix. This was also fixed in Tokio
1.x-compatible versions of Hyper. There are still users of hyper-02 in fbcode.
However, this is only used for servers and only when accepting websocket
connections, and those users are just using Hyper as a HTTP client.

Reviewed By: farnz

Differential Revision: D28091331

fbshipit-source-id: de13b2452b654be6f3fa829404385e80a85c4420
2021-04-29 08:07:45 -07:00
Thomas Orozco
ffed22260d third-party/rust: remove Gotham 0.2
Summary:
This used to be used by Mononoke, but we're now on Tokio 1.x and on
corresponding versions of Gotham so it's not needed anymore.

Reviewed By: farnz

Differential Revision: D28091091

fbshipit-source-id: a58bcb4ba52f3f5d2eeb77b68ee4055d80fbfce2
2021-04-29 08:07:45 -07:00
svcscm svcscm
46354e4c17 Updating submodules
Summary:
GitHub commits:

d85ae094eb
8f236c95f1
bb8f85bf21
60c8005870
e05c431f55
85becd94c1
00ff6f4709
e6019b317f

Reviewed By: jurajh-fb

fbshipit-source-id: 7f9907ce23ab5ea6710a8e2cc4e2989e56b45538
2021-04-29 07:49:15 -07:00
Alex Hornby
40a7e12e56 mononoke: make scrub EmptyRoute produce WalkKeyOptPathHash
Summary:
Connect up the scrub stream types so they will be uniform for scrubs that log pack info and those that do not.

This is in preprepation for the next diff which connects up the pack info logging of path hashes to scrub.    CI for this diff verifies its not broken the non-path tracking case.

Differential Revision: D28031868

fbshipit-source-id: 7bf91eb1778f57487f6a2847f215cf7f5cd2dff7
2021-04-29 07:46:29 -07:00
Alex Hornby
58f52f6422 mononoke: make WalkKeyOptPath generic over path and pathhash
Summary: This moves evolve_path up to WrappedPathLike so that we can use sample route evolution logic for routes that track paths (e.g. corpus sampling) and path hashes (e.g. scrub, where path hashes take less memory than full paths).

Differential Revision: D28031867

fbshipit-source-id: cdabdc466158a8db1c770536747c996dddb27e71
2021-04-29 07:46:29 -07:00
Alex Hornby
b296a73c2c mononoke: name the fields in WalkPayloadMtime
Summary: Name the fields rather than leave it as a tuple struct.  This makes it a bit easier to work with in the rest of the stack

Differential Revision: D28062254

fbshipit-source-id: 9e5202b4d6f1f29d44d98b86aa9b6ddb97d821eb
2021-04-29 07:46:29 -07:00
Alex Hornby
ff462f5422 mononoke: move filter_repo_path to be a method on NodeType
Summary: Makes more sense for this to be a method on NodeType

Differential Revision: D28031869

fbshipit-source-id: 1ddbafa0d7634ac67fd8d5112e6f57759ed91638
2021-04-29 07:46:29 -07:00
Alex Hornby
4e7f0888bf mononoke: name the fields in WalkKeyOptPath
Summary: Name the fields rather than leave it as a tuple struct

Differential Revision: D28031866

fbshipit-source-id: 039f004e0b81294aa6d6b13e79cb45ee2b84567c
2021-04-29 07:46:29 -07:00
Alex Hornby
901fe40f5f mononoke: introduce WrappedPathLike to walker
Summary: This new trait abstracts across WrappedPath and WrapperPathHash.  Later in the stack I make path tracking use this to track either full paths (for corpus sampling) or path hashes (for logging from scrub).

Differential Revision: D28031870

fbshipit-source-id: d1c57230f68fffff179929a3cb92c82d92e0588c
2021-04-29 07:46:29 -07:00
Thomas Orozco
71994ae65d mononoke: be less specific in errors we expect in test-lfs-server-scuba-logging.t
Summary:
Like it says in the title. This isn't giving us the same error consistently
causing flaky failures.

Reviewed By: StanislavGlebik

Differential Revision: D28091747

fbshipit-source-id: dfc7a28b443c6577823c71cee7b006ed30fec18e
2021-04-29 06:44:49 -07:00
svcscm svcscm
f8000eeb86 Updating submodules
Summary:
GitHub commits:

65bfdbdb7f

Reviewed By: jurajh-fb

fbshipit-source-id: 95bf8440f7c52ad0c381c5eb5c11797a4a57e77a
2021-04-29 06:19:17 -07:00
Mark Juggurnauth-Thomas
a32ccfc5cc blobrepo: delete BlobRepo::new_dangerous
Summary: This is no longer needed, as all construction is performed by facet factories.

Reviewed By: StanislavGlebik

Differential Revision: D28001390

fbshipit-source-id: 237dd4f7b8b08bec5b85360edc3be7018d9161de
2021-04-29 06:11:20 -07:00
Mark Juggurnauth-Thomas
139d93bedb changesets: split implementation to a separate crate
Summary:
Keeping the `Changesets` trait as well as its implementations in the same crate means that users of `Changesets` also transitively depend on everything that is needed to implement it.

Flatten the dependency graph a little by splitting it into two crates: most users of `Changesets` will only depend on the trait definition.  Only the factories need depend on the implementations.

Reviewed By: krallin

Differential Revision: D27430612

fbshipit-source-id: 6b45fe4ae6b0fa1b95439be5ab491b1675c4b177
2021-04-29 06:11:20 -07:00
Mark Juggurnauth-Thomas
d66e56c407 changesets: remember repo_id in changesets
Summary:
The changesets object is only valid to access the changesets of a single repo
(other repos may have different metadata database config), so it is pointless
for all methods to require the caller to provide the correct one.  Instead,
make the changesets object remember the repo id.

Reviewed By: krallin

Differential Revision: D27430611

fbshipit-source-id: bf2c398af2e5eb77c1c7c55a89752753020939ab
2021-04-29 06:11:20 -07:00
Mark Juggurnauth-Thomas
b935836e32 changesets: replace get_sql_changesets with enumeration methods
Summary:
The `get_sql_changesets` method on `Changesets` is an abstraction violation,
and prevents extraction of `SqlChangesets` to a separate crate as it would
introduce a circular dependency.

It is used to allow bulk queries to enumerate changesets by integer unique ID,
so promote this to a full feature of `changesets`, and remove the
`get_sql_changesets` method.

Reviewed By: krallin

Differential Revision: D27426921

fbshipit-source-id: 2839503029b262dd5e6a8be09bb35bb143b4c5ac
2021-04-29 06:11:20 -07:00
svcscm svcscm
cdeaae40c2 Updating submodules
Summary:
GitHub commits:

4582c9d90d
fe2d29abc6
bfb1f93067
224e93ee9c
58567289f8
c7df5c045d

Reviewed By: jurajh-fb

fbshipit-source-id: d225a14c2f00f491ea4317cbdbb111844b65cad4
2021-04-29 05:38:40 -07:00
svcscm svcscm
bffc218e04 Updating submodules
Summary:
GitHub commits:

67b4e543bb
9fbf6364a0
7d8b799fba
05344317b3
a4be575e69

Reviewed By: jurajh-fb

fbshipit-source-id: 89d1f9b170a4ffae5761947ab82bf22812841fc7
2021-04-29 03:21:10 -07:00
svcscm svcscm
716a665543 Updating submodules
Summary:
GitHub commits:

0d927e0f7c
d58f87cd48
192432c203
8e7aa3f2c2
d73aa074c4
273257d6f9

Reviewed By: jurajh-fb

fbshipit-source-id: 29910bf41a26ffd207540a234b89bb08164d1970
2021-04-29 02:18:06 -07:00
svcscm svcscm
12a9678a7b Updating submodules
Summary:
GitHub commits:

1f48717717
b84e18e5be
bbb8c0f97b
5efbebbdd8
ab82698c2b
0a7e24d669
e6cf80618d
637e31003e
1943d4047b

Reviewed By: jurajh-fb

fbshipit-source-id: 970b5b68699bc21b07b9b2355f5f013aa3be9df0
2021-04-29 01:35:46 -07:00
svcscm svcscm
1bf85935a3 Updating submodules
Summary:
GitHub commits:

c8b1d4182b
a8fbfdf1c1
1e10e458f4
bb9134d252
797d541796
d0d64ca7a6
1db393a91f
24faa1a8c0
70ab2b5ee4
351eadb947

Reviewed By: jurajh-fb

fbshipit-source-id: 59b963f7d439c31ccb2374443f62d1caee1b82af
2021-04-28 23:56:10 -07:00
svcscm svcscm
f3f1f82fb0 Updating submodules
Summary:
GitHub commits:

0b19393baf

Reviewed By: jurajh-fb

fbshipit-source-id: 564db0a5e5766915f5516633f5eb2372fc53b49e
2021-04-28 23:06:11 -07:00
svcscm svcscm
01b67acb96 Updating submodules
Summary:
GitHub commits:

4d5ebf1602
9a84a03beb
11dfb69937
68c0d0bd4a

Reviewed By: jurajh-fb

fbshipit-source-id: 6b749c3a5030d5504cf303559b2943cf8e4c1836
2021-04-28 22:04:52 -07:00
svcscm svcscm
65aa4bab7c Updating submodules
Summary:
GitHub commits:

1046ed6779
de97c5e4fd
df40ab771e
7d25576e09
9c505c8afd
365edc2402
6ab8873c07

Reviewed By: jurajh-fb

fbshipit-source-id: 2d3dae5fe972fe241a3f994993174851bdc7ea7d
2021-04-28 21:21:58 -07:00
svcscm svcscm
2e3ab3d387 Updating submodules
Summary:
GitHub commits:

b98c2d4700
44b457e1b9
8c3994c213
fddf4bac1e
4a9f2cdd4d
f9df3a0c25
237d614191
e002140ee6
458b89986b

Reviewed By: jurajh-fb

fbshipit-source-id: fefb5a0b0d687c7fd18ce64fdad56023c4427b06
2021-04-28 18:31:58 -07:00
svcscm svcscm
16df1caaa2 Updating submodules
Summary:
GitHub commits:

3a47a97951
916de2270b
ec89079c7d
a5a30d7342
481504cf2e
b0e6c7856a
bcafb639aa
e9d080fef9
06d58087a2
82cb5439f4

Reviewed By: jurajh-fb

fbshipit-source-id: 8af9bd36cbc6524f643485d0a87dfcde687dbcaa
2021-04-28 17:46:16 -07:00
svcscm svcscm
d4228a866e Updating submodules
Summary:
GitHub commits:

2a80218d46
e093c49f75

Reviewed By: jurajh-fb

fbshipit-source-id: 840a9cd01e93a5b218044ccffbfd7fa3409686aa
2021-04-28 17:10:09 -07:00
Xavier Deguillard
42b17dfa1c nfs: add to Executor directly without creating a future
Summary:
folly::via is a Future API, and thus it creates one, which requires allocating
it and then attaching it to the Executore. Since the code to dispatch a request
isn't Future based, we don't need to use folly::via, and we can simply add the
lambda to the Executor directly. This removes expensive memory allocations from
the EventBase.

Reviewed By: kmancini

Differential Revision: D27976674

fbshipit-source-id: 8fa9724a94ba69b071ab894cdbbad0d33733c098
2021-04-28 17:06:23 -07:00