Commit Graph

62659 Commits

Author SHA1 Message Date
Liubov Dmitrieva
14a0d96b04 add deprecation notice for pullbackup related set of commands
Summary:
this is needed to remind users to use the commands related to the Commit Cloud Workspaces

This is also needed to avoid confusion - the current error is not very informative:

```
connected to twshared9522.24.frc3.facebook.com session Wfq9tGuoPSBJH8nc
abort: 'listkeyspatterns' command is not supported for the server ssh://hg.vip.facebook.com//data/scm/fbsource?infinitepush_write
```

Reviewed By: markbt

Differential Revision: D26073439

fbshipit-source-id: 58ee477bc021a0796e22cc18610225ee003e06be
2021-01-26 08:21:07 -08:00
Thomas Orozco
ad5dc2d59d third-party/rust: hyper-02: update to 0.13.9
Summary:
Just a minor version update. I'd like to add a patch on top of this for a PR
that hasn't been merged yet, but updating to the underlying released version
first will make the diff clearer.

Reviewed By: ahornby

Differential Revision: D26047997

fbshipit-source-id: 91856f645ec3aaaf4fbf256a23c7e8d4db0f6b37
2021-01-26 07:21:28 -08:00
Liubov Dmitrieva
2c15a29f0f fix unit test that was broken by D19803760
Summary: fix unit test that was broken by D19803760 (8e35bcc49d) to unblock release

Reviewed By: krallin

Differential Revision: D26073002

fbshipit-source-id: 88da60f5806ef5cb2f832922e503309836542e09
2021-01-26 04:33:47 -08:00
svcscm
432cef9395 Updating submodules
Summary:
GitHub commits:

abb2812be7

Reviewed By: wittgenst

fbshipit-source-id: 8084f2362decab720cd226e7bddfd5a0fe69d11d
2021-01-26 04:33:47 -08:00
svcscm
26ddb10827 Updating submodules
Summary:
GitHub commits:

91814ec04c
12f1137355
85165a158e

Reviewed By: wittgenst

fbshipit-source-id: ec0084104d834d837d221247167b04fc34b7a867
2021-01-25 22:58:27 -08:00
Daniel Xu
5715e58fce Add version specificiation to internal dependencies
Summary:
Lots of generated code in this diff. Only code change was in
`common/rust/cargo_from_buck/lib/cargo_generator.py`.

Path/git-only dependencies (ie `mydep = { path = "../foo/bar" }`) are not
publishable to crates.io. However, we are allowed to specify both a path/git
_and_ a version. When building locally, the path/git is chosen. When publishing,
the version on crates.io is chosen.

See https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#multiple-locations .

Note that I understand that not all autocargo projects are published on crates.io (yet).
The point of this diff is to allow projects to slowly start getting uploaded.
The end goal is autocargo generated `Cargo.toml`s that can be `cargo publish`ed
without further modification.

Reviewed By: lukaspiatkowski

Differential Revision: D26028982

fbshipit-source-id: f7b4c9d4f4dd004727202bd98ab10e201a21e88c
2021-01-25 22:10:24 -08:00
svcscm
014b7625da Updating submodules
Summary:
GitHub commits:

dfabe4ce46
1ab6a01f76

Reviewed By: wittgenst

fbshipit-source-id: 71ff6e03b4a63dfdeeab958ceba64088869461a9
2021-01-25 22:10:23 -08:00
svcscm
c2244fbbe0 Updating submodules
Summary:
GitHub commits:

a227d297d8
3c764a2d6c

Reviewed By: wittgenst

fbshipit-source-id: 5f424753571aa53f1125abcff6073d79023d1b27
2021-01-25 18:39:20 -08:00
svcscm
5ea945353b Updating submodules
Summary:
GitHub commits:

938cef3500
fe5710128d
b69bbdf0e2

Reviewed By: wittgenst

fbshipit-source-id: 503cd629478a0ace9379199ad2751454db225bb1
2021-01-25 18:39:20 -08:00
svcscm
e73984b49b Updating submodules
Summary:
GitHub commits:

c29293df0b
0fd2fc2fae
7c6611cbdb
1316e8c3bd

Reviewed By: wittgenst

fbshipit-source-id: 59088052e88d3d87392191b3e25189af65846a21
2021-01-25 16:38:51 -08:00
Genevieve Helsel
9db3bf1e25 run linter in eden/fs/py
Summary: just running the linter :)

Reviewed By: chadaustin

Differential Revision: D26000280

fbshipit-source-id: 23bc996ca361b208cd6f643e1440eb2a9aeec034
2021-01-25 16:13:55 -08:00
Genevieve Helsel
7565ff7f59 run linter in eden/fs/docs
Summary: Just running the linter :)

Reviewed By: singhsrb

Differential Revision: D26000274

fbshipit-source-id: 5d94abf11210fda5e956c408764aa0c348aa0d84
2021-01-25 16:13:54 -08:00
Genevieve Helsel
55da8ffcbc run linter in eden/integration
Summary: just running the linter!

Reviewed By: chadaustin

Differential Revision: D26000269

fbshipit-source-id: 184eb962fa7b88eb9b8b6bd22ae76477cbb6a06c
2021-01-25 16:13:54 -08:00
svcscm
ba6971a797 Updating submodules
Summary:
GitHub commits:

07a3a628d5
88382fbfe7
efadee9e20
1d226018af
ed1f0417c7

Reviewed By: wittgenst

fbshipit-source-id: 611077d2ea17b264ed9c147833ae2585f88d2e0b
2021-01-25 16:13:54 -08:00
svcscm
8b91f01b7f Updating submodules
Summary:
GitHub commits:

7b67e06762
3188750513
3657644b0f
88547cb221
431e8afba7
7f1185f4cd
8bb14ecd8d
757eff8093
33c87bb7b6
fd40801554
f243a25318
3e56221374

Reviewed By: wittgenst

fbshipit-source-id: e0b74e240f732addd4f1330560429195661cf536
2021-01-25 14:16:40 -08:00
Stanislau Hlebik
fa5b61f577 mononoke: add rpm for megarepotool
Summary: I'd like to install it on scm devservers, and that's the easiest way to do it.

Reviewed By: HarveyHunt

Differential Revision: D26048848

fbshipit-source-id: d463f99b710b60b34a06fb0bc8cbe2af9fe418f2
2021-01-25 13:30:08 -08:00
svcscm
4d166211c7 Updating submodules
Summary:
GitHub commits:

2d173fe622
f906ccd95d
b1e069e477
4731bb0c37
2b96889a0f
55c183ba85
ad2415d3b7
ed5fa7611b
d43298f2cb
767af9fad6
5f6279b2a7
57bfe6497c
5ab476fe0c

Reviewed By: wittgenst

fbshipit-source-id: f62aeff0d066927b967ac855e469042c4670ed17
2021-01-25 13:17:10 -08:00
Chengxiong Ruan
85e4390166 Fix weird looking highlighted values in below (#8074)
Summary:
Pull Request resolved: https://github.com/facebookincubator/resctl/pull/8074

1. Upgrade `Cursive` to v0.16.3
2. patch `cursive_buffered_backend` with a temp fork before v0.4.2 is released. This is needed to avoid Rust being confused by `Backend` trait from different versions of `cursive_core`

Reviewed By: dschatzberg

Differential Revision: D25987535

fbshipit-source-id: 6826653f9b6d4d6bdba485abf42eb517a853acc1
2021-01-25 13:12:37 -08:00
Chad Austin
4196d44f5e include FUSE result codes in eden strace
Summary: Like `strace`, show the return codes from FUSE requests.

Reviewed By: kmancini

Differential Revision: D26033195

fbshipit-source-id: 2347129ce480e50a3b0f588937e535e0df45dfbd
2021-01-25 12:35:36 -08:00
Chad Austin
73b0d2a4a4 log the name and mode for FUSE_CREATE requests
Summary:
Correct an oversight that did not show the name of the created file in
a FUSE_CREATE request.

Reviewed By: kmancini

Differential Revision: D26033166

fbshipit-source-id: 35eb8a844aff29b519318109c918f2a700692835
2021-01-25 12:35:36 -08:00
Chad Austin
e282452d65 always write log files as UTF-8 to avoid cp1252 encoding errors
Summary:
The log file should be opened with a defined encoding rather than
whatever the platform default is. On Windows, that might be cp1252,
which throws an encoding error when the text is unicode.

Reviewed By: genevievehelsel

Differential Revision: D26025779

fbshipit-source-id: 0102b2555e91812dfe94a7c332eb896ca2c9ef7e
2021-01-25 12:35:36 -08:00
svcscm
38dc12d95b Updating submodules
Summary:
GitHub commits:

bf66d6275b
f313b0238d

Reviewed By: wittgenst

fbshipit-source-id: f9353dd1aa1898cf5af124aef6557b2d11a63608
2021-01-25 11:00:28 -08:00
Alex Hornby
217c07c65a mononoke: introduce tail params to walker
Summary:
We are going to pass more params into tail.rs as part of being able to clear state between chunks.

This prepares by creating a struct for them and adding command line args for include/exclude node types to clear.

Reviewed By: krallin

Differential Revision: D25910615

fbshipit-source-id: 610a884c17da7af1e23cfa81d4f495fe03bad9a3
2021-01-25 09:11:07 -08:00
svcscm
e06f3b3f19 Updating submodules
Summary:
GitHub commits:

86730ac964

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: edfdac2ebbc6123f31a923c3e7b8d2fbb741caca
2021-01-25 08:55:28 -08:00
Thomas Orozco
4dd3461824 third-party/rust: update Tokio 0.2.x to 0.2.24 & futures 1.x to 1.30
Summary:
When we tried to update to Tokio 0.2.14, we hit lots of hangs. Those were due
to incompatibilities between Tokio 0.2.14 and Futures 1.29. We fixed some of
the bugs (and others had been fixed and were pending a release), and Futures
1.30 have now been released, which unblocks our update.

This diff updates Tokio accordingly (the previous diff in the stack fixes an
incompatibility).

The underlying motivation here is to ease the transition to Tokio 1.0.
Ultimately we'll be pulling in those changes one or way or another, so let's
get started on this incremental first step.

Reviewed By: farnz

Differential Revision: D25952428

fbshipit-source-id: b753195a1ffb404e0b0975eb7002d6d67ba100c2
2021-01-25 08:06:55 -08:00
Ilia Medianikov
7370db0552 Make it possible to filter bookmark log entries by timestamp
Summary:
Add `start-time` and `end-time` arguments to `mononoke_admin bookmarks log` command.

Example: `mononoke_admin bookmarks log master --start-time 2h --end-time 1h` to log entries that are from 2 to 1 hour old.

Reviewed By: StanislavGlebik

Differential Revision: D26001141

fbshipit-source-id: 1ad2f9347a4414800ee4830961df188b447a673a
2021-01-25 05:26:13 -08:00
Stanislau Hlebik
8aac4fb598 mononoke: use fsnodes while verifying cross-repo working copy
Summary:
Previously we were using two step process of verifyng working copy - we fetched
all manifests and compared filenodes. If filenodes are different we also
fetched content ids. This was harder to read and it also was slower.

Instead let's use fsnodes instead - they can give filetypes and content ids
directly. Also move all filenode related code to commit_validator, since it's the only place that uses it

Reviewed By: krallin

Differential Revision: D26020705

fbshipit-source-id: 9c598b4ac4950c316195dd10952ee0a785854538
2021-01-25 05:12:52 -08:00
Stanislau Hlebik
a53a03d9bf mononoke: make two functions generic in cross repo validation
Summary:
I'm going to change how we do working copy validation in the next diffs,
however these function will still be useful. So let's keep them.

Reviewed By: krallin

Differential Revision: D26020704

fbshipit-source-id: 32f527cf821de692529d2d23a3cb4b7e90940a32
2021-01-25 05:12:52 -08:00
Jan Mazur
83483c65bf debugnetwork should check HTTP response status codes
Summary:
Without checking response status codes, we don't really know whether our request got processed as expected.

I am also adding port to url because default path doesn't have it. HTTPS is 443 by default.

Reviewed By: johansglock

Differential Revision: D26046197

fbshipit-source-id: c6082cc221c44c6c06557bccaaebb53efd56b4bf
2021-01-25 04:56:07 -08:00
Stanislau Hlebik
960c9943ba mononoke: tweak how we decide if a generation number is low or not
Summary:
A bit of background: some time ago I landed an optimization to getbundle, which
 split the commits user want to fetch into "high generation numbers"
(which means commits are close to main bookmark) and "low generation numbers"
(commits that are far away from the main bookmark). User can fetch "low
generation numbers" if e.g. a commit to an old release branch was landed.
Processing them separately can yield significant speedups.

Previously we chose a threshold statically e.g. if a commit has generation
number lower than X then it's considered to have a low generation number.
Threshold was chosen arbitrarily and relatively small [1], and it didn't work
that well for commits that were above this threshold, but still not close
enough to main bookmark - we still see cpu spikes.

Instead let's define a commit as having low generation number if it's >X
commits farther from the commit with the highest generation number.

Reviewed By: ahornby

Differential Revision: D25995936

fbshipit-source-id: 57eba4ba288114f430722266a8326183b3b1f0bd
2021-01-25 03:37:31 -08:00
Stanislau Hlebik
51aaf8eade mononoke: incapsulate logic of checking low gen number
Summary:
In the next diff i'm going to tweak how we decide if low generation number is
low or not [1]. To make it easier to add this tweak let's put the logic of
figuring out low generation number in a struct.

[1] If generation number is low then we run a few heuristics to speed up the
getbundle request

Reviewed By: ahornby

Differential Revision: D25995938

fbshipit-source-id: dbb95b4321d5a4caa13c4183882e90b23020503c
2021-01-25 03:37:31 -08:00
Jan Mazur
c51f37ffbd speed test on mononoke
Summary: Using speedtest http endpoint on mononoke when client is connecting directly to mononoke.

Reviewed By: markbt

Differential Revision: D25899297

fbshipit-source-id: b9e1fb90080c42cb2bd242a1cb3612c63a8f93a4
2021-01-25 02:29:19 -08:00
svcscm
dff68e5bca Updating submodules
Summary:
GitHub commits:

c3caeac617
97e1256314
29e283aaa0
b6c3d4f286
259d212da0

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 3a23d8bc8c72dd9428d5f52e03c33053b6e53912
2021-01-25 02:29:19 -08:00
svcscm
ac38d57a77 Updating submodules
Summary:
GitHub commits:

a8503e783f
1ac133996f

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: e1783c64be97ef06b07d0ec81f4c4f3663f67033
2021-01-24 20:23:39 -08:00
svcscm
8c9c3e55d6 Updating submodules
Summary:
GitHub commits:

ad4412122c
124ecc8bfd
d2bd82b975
e0fdc51d30
d5e3666551

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: f7e34afec384f590523060207d331c0bd55da3bc
2021-01-24 11:23:21 -08:00
svcscm
e4db35ad28 Updating submodules
Summary:
GitHub commits:

9c474254f6

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 40a5a076ce77ed3f1ebba34bf59592884e09c181
2021-01-23 10:50:54 -08:00
svcscm
493086cef8 Updating submodules
Summary:
GitHub commits:

f5b3aa4dfc
aeb4608644
19f75009ec
1406133520
a58e56ee99
e420569497
8f8c2e8bee
884584d7f2

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 487a1ce23f7cd27804226c109bb332a65161db05
2021-01-23 10:50:54 -08:00
svcscm
968641214c Updating submodules
Summary:
GitHub commits:

94ddf6eac7
da2df0c757
d72058aac7

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 87e1c97280dab29c558d289ec2ff439826f51b84
2021-01-22 13:51:31 -08:00
Jun Wu
8e35bcc49d tracing-analyzer: log version information
Reviewed By: DurhamG

Differential Revision: D19803760

fbshipit-source-id: 802d72c1a7aab495d2b6aecd2f8bb696ce2fc522
2021-01-22 12:31:16 -08:00
Jun Wu
7656b5796e tracing-analyzer: aggregate logging data from tracing data
Summary:
This new crate is part of the new telemetry / logging effort.  Its input is
tracing data, and output is aggregated NoSQL table content.

This diff is only the start, setting up the direction.

Reviewed By: DurhamG

Differential Revision: D19797702

fbshipit-source-id: bdf34461c05b5eae5e59652bc82d8ee1857dbf1e
2021-01-22 12:31:16 -08:00
svcscm
53b7c7e248 Updating submodules
Summary:
GitHub commits:

3657c3296e
b0dfa5dc35
308822a7b8

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: af5de389ee7b7fb41ebba4eb08b966b8632b105b
2021-01-22 12:31:15 -08:00
svcscm
5387aecc7b Updating submodules
Summary:
GitHub commits:

7e2f5928df
1ea2e153e0
e0a0d3e3e6

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: de99ce88d40089d6cf96452f90ecd2b2a6cbf501
2021-01-22 10:18:26 -08:00
Ilia Medianikov
bb9a4e5e8c Record blob size to blobstore healer queue
Summary:
Schema for `xdb.mononoke_blobstore_sync_queue : blobstore_sync_queue` wasn't altered yet as AOSC [wasn't working](https://fb.workplace.com/groups/mysql.users/permalink/4991036464278262/).

This will allow the blobstore healer to make decisions about how many blobs it should copy at once. The goal is to eliminate the OOMs that were regularly seen by giving the blobstore healer a memory budget.

Reviewed By: krallin

Differential Revision: D25925522

fbshipit-source-id: d3714dbadc74274a4c4a0e66fa732b84bef89227
2021-01-22 08:34:16 -08:00
Jan Mazur
d1b291d72a flush stream to always print out interactive prompt
Summary: Suddenly prompt stopped appearing for me. Flush the stream to be sure that it's printed out.

Reviewed By: HarveyHunt

Differential Revision: D25956018

fbshipit-source-id: 83419037fa6ce672e203385b71f1403a738d0c90
2021-01-22 07:46:11 -08:00
Alex Hornby
799203c8fa mononoke: remove ThrottleOptions::new, make struct instance instead
Summary: Its easier to distinguish the parameters when constructing it directly rather than via a new() method.

Differential Revision: D26017347

fbshipit-source-id: a020db1133de727f217f67a05953059122e3623a
2021-01-22 07:24:14 -08:00
Alex Hornby
f2e3367985 mononoke: move add_blobstore_args into MononokeAppBuilder
Summary: Its only called from inside MononokeAppBuilder so move it inside to save passing all the struct members as params.

Differential Revision: D25976405

fbshipit-source-id: e95d7b8f5f4474f0289d29bb7bb0a8b0780112e0
2021-01-22 07:24:14 -08:00
Liubov Dmitrieva
a1dd7229e6 dynamic config: remove unnecessary option from the dynamic config
Summary: this infinitepush option is true by default

Reviewed By: DurhamG

Differential Revision: D25995928

fbshipit-source-id: 0c1d50a96dc16ee4d8c45da4f4a213baa49bb8a4
2021-01-22 06:24:23 -08:00
Alex Hornby
a5d86bdae9 mononoke: use strum for ScrubAction parsing
Summary: Removes the manual from_str impl

Reviewed By: krallin

Differential Revision: D25944622

fbshipit-source-id: 4e1e1113e0501c01435b287b165c0f1695729615
2021-01-22 06:04:54 -08:00
Alex Hornby
5285dd1b3c mononoke: move enum ScrubAction to multiplexedblob
Summary: This doesn't need to be in metaconfig anymore, can move it to multiplexedblob

Reviewed By: krallin

Differential Revision: D25928061

fbshipit-source-id: 8aa6ce6aafa16f84730cf388ebf7eab6d5bf2c53
2021-01-22 06:04:54 -08:00
Stanislau Hlebik
7770133dbe mononoke: add more getbundle logging
Summary:
We have a few tricky opimizations, so it's better to have more logging than
less.

Reviewed By: HarveyHunt

Differential Revision: D25995937

fbshipit-source-id: b5502708125b70f3d656be3dc1120176f5c76ce8
2021-01-22 05:40:08 -08:00