Commit Graph

57729 Commits

Author SHA1 Message Date
Mohan Zhang
2ef3e20e4d Run auto cargo locally
Summary: Since thirdpart depedency change on D21341319

Reviewed By: jsgf, wqfish

Differential Revision: D21417890

fbshipit-source-id: 3cc6bafa23512c7ae489513216bcafa46e7a744f
2020-05-05 20:59:02 -07:00
Chad Austin
0e86e57c2c disallow import helpers from spawning watchman instances
Summary:
During a graceful restart, while the fuse mount is not handling
requests, avoid a possible deadlock between the edenfs, hg
debugedenimporthelper, and watchman processes.

See the comment in HgImporter.cpp.

Reviewed By: fanzeyi

Differential Revision: D21342275

fbshipit-source-id: df8fb5df5d5cd1490e88b42054b34cbb2acdb692
2020-05-05 20:46:31 -07:00
Zeyi (Rice) Fan
952069397c fix get blob local
Summary: This bug got in while iterating the original Diff. It should only be returning empty when the blob does not exist locally.

Reviewed By: xavierd

Differential Revision: D21417659

fbshipit-source-id: 676e22313ab4a024af5341d8c99797fc062bd293
2020-05-05 20:21:21 -07:00
svcscm
ffc1ff373a Updating submodules
Summary:
GitHub commits:

ca3fc479ed
c21c459771
7ab262586c
be20f1c25c

Reviewed By: jurajh-fb

fbshipit-source-id: 03c4364247da35b714e50c2f0253f9bb0c63e77f
2020-05-05 20:21:21 -07:00
svcscm
377be3ddbb Updating submodules
Summary:
GitHub commits:

6ad92107a5
911311aba9
11f49c1469
ca296a862e
481d507212
354bb79e06
fb0e5fd94b
84c9b9c565

Reviewed By: jurajh-fb

fbshipit-source-id: 25f72385b08f151b4d2af9bb65d7af410842ee18
2020-05-05 19:00:09 -07:00
Durham Goode
022196d3b7 configs: convert instagram-server_overrides.rc
Summary: Converts instagram-server_overrides.rc to our dynamic config generator

Reviewed By: quark-zju

Differential Revision: D21339855

fbshipit-source-id: 1cd4709c1f239322de3ccf516c4ebef519ea41ac
2020-05-05 18:19:11 -07:00
Durham Goode
726e3f6a73 configs: convert configerator_overrides.rc
Summary: Converts configerator_overrides.rc to our dynamic config generator

Reviewed By: quark-zju

Differential Revision: D21339856

fbshipit-source-id: 84bd17d04db1a2e6070eb5e75bda668a596822d4
2020-05-05 18:19:11 -07:00
Durham Goode
97d84e3b5d configs: move hgrc.dynamic to always be in the shared repo
Summary:
Instead of trying to maintain two hgrc.dynamic's for shared repositories,
let's just always use the one in the shared repo. In the long term we may be
able to get rid of the working-copy-specific hgrc entirely.

This does remove the ability to dynamically configure individual working copies.
That could be useful in cases where we have both eden and non-eden pointed at
the same repository, but I don't think we rely on this at the moment.

Reviewed By: quark-zju

Differential Revision: D21333564

fbshipit-source-id: c1fb86af183ec6dc5d973cf45d71419bda5514fb
2020-05-05 18:19:10 -07:00
Durham Goode
c326c5fc6b configs: log dynamic config mismatches to scuba
Summary: Let's log the mismatches to scuba so we can track them down.

Reviewed By: quark-zju

Differential Revision: D21313255

fbshipit-source-id: ee79c9504a80ebe5b21849c0eae5993b65eaff28
2020-05-05 18:19:10 -07:00
Durham Goode
b9223b9db6 sampling: move ui.log sampling logic to core
Summary:
In a future diff we want to log information about configuration
mismatches. In order to do that, we need to be able to call ui.log before
extensions are setup. Let's move the ui.log sampling logic into core so it can
be called before extension setup.

Reviewed By: quark-zju

Differential Revision: D21313257

fbshipit-source-id: fe1c0f572720c17e7398f2a4fa7082ef8fb59536
2020-05-05 18:19:09 -07:00
Durham Goode
ece8408993 configs: migrate devserver_no_sandcastle_overrides.rc
Summary:
Moves the conditional configuration from
devserver_no_sandcastle_overrides.rc into our dynamic config fb rules.

Reviewed By: quark-zju

Differential Revision: D21310916

fbshipit-source-id: 116711afbfa9fb9f134bbe55b7b8b302b39dab36
2020-05-05 18:19:09 -07:00
Durham Goode
e67d609e1d configs: validate dynamic configs
Summary:
Adds python logic for validating the dynamic configs. Any dynamic
configs that don't match the given list of rc files will be reported and removed

Reviewed By: quark-zju

Differential Revision: D21310919

fbshipit-source-id: 07f584bba990f1b01347dfbc285e3ca814fe5c5a
2020-05-05 18:19:09 -07:00
Durham Goode
6e7f85b949 config: load .hg/hgrc.dynamic
Summary:
Adds .hg/hgrc.dynamic to the default load path, before .hg/hgrc though,
so it can be override.

Reviewed By: quark-zju

Differential Revision: D21310921

fbshipit-source-id: 288a2a2ba671943a9f8532489c29e819f9d891e1
2020-05-05 18:19:08 -07:00
Durham Goode
dc90e2ca04 configs: add domain, platform, and improve dynamicconfigs
Summary:
Adds dynamic config conditionals for domain, platform, and adds a few
useful helper functions.

Reviewed By: quark-zju

Differential Revision: D21310920

fbshipit-source-id: 58f35e52d6d7a4edae2b3aefff533ef2c021aa57
2020-05-05 18:19:08 -07:00
Xavier Deguillard
8f872412e6 config: restrict stat comparison on Windows
Summary:
On Windows, the following pseudo code:

  int fd = open("file");
  struct stat st;
  fstat(fd, &st);

Will have a different st_dev than the following code:

  struct stat st;
  stat("file", &st);

Since the FileChangeMonitor uses st_dev as a way to compare if a file
changed, the config is always reloaded.

For our use case, the filesize and its mtime should be enough to know if the
configuration changed, so let's only use these 2 on Windows.

Reviewed By: wez

Differential Revision: D21312679

fbshipit-source-id: f08b3eb7d6037f5d88ece82efe3a5437b1954ba2
2020-05-05 18:14:55 -07:00
Xavier Deguillard
5fc68d15f9 inodes: enable RenameTest on Windows
Summary:
Only the mtime test had to be disabled due to getMetadata not being tracked
on Windows.

Reviewed By: wez

Differential Revision: D21312680

fbshipit-source-id: 5678956ed2b8b45b38d44c459557161aa6fd222e
2020-05-05 18:14:55 -07:00
Xavier Deguillard
6d726d574a inodes: enable RemoveTest for Windows
Summary:
Fairly straightforward, one change to the test was necessary as reading an
unlinked file cannot be done on Windows.

Reviewed By: wez

Differential Revision: D21312681

fbshipit-source-id: c8cca5eeca7825983176ef618007c514b8e02140
2020-05-05 18:14:54 -07:00
Xavier Deguillard
6d4a55a3ea win: change the argument order for writeFile
Summary:
This brings it closer to folly::writeFile which should help in avoiding ifdef
whenever we want to use it.

Reviewed By: wez

Differential Revision: D21319020

fbshipit-source-id: 80fbf7fba671b18b5ef68375910e1a2a8869f590
2020-05-05 18:14:54 -07:00
svcscm
a5d3bbc25e Updating submodules
Summary:
GitHub commits:

c230732e3f
2829163852
32215b0c25
394f2bbd13

Reviewed By: jurajh-fb

fbshipit-source-id: 72de1d434eee60afd38134ad367d452c69d3494f
2020-05-05 15:48:06 -07:00
Durham Goode
8744b81151 git: update git dependency to 0.13.5 to match internal version
Summary:
Our internal git dependency got upgraded, so we need to upgrade our
Cargo.toml version.  Unfortunately this doesn't seem to have any test coverage?

Reviewed By: singhsrb

Differential Revision: D21410241

fbshipit-source-id: 64fe7f39a9c93aa5d97ce095ee1641c1cc6ed365
2020-05-05 15:35:12 -07:00
Adam Simpkins
2e0f442a24 work around slightly broken AF_UNIX connect() behavior on Windows
Summary:
On Windows calling `connect()` on an AF_UNIX socket path that does not exist
creates a file at that location.  This is problematic as it now prevents
servers from binding to that path.  Even if the server attempts to remove the
file in order to bind, clients attempting to call `connect()` can race with it
and make binding fail.

This updates our client connection code check to see if the file exists before
attempting to call `connect()`.  This can still race with a server that is
trying to remove an old socket and re-bind, but it at least makes the race
less likely to happen.

Reviewed By: genevievehelsel

Differential Revision: D21410571

fbshipit-source-id: 3df63b19b40b25be98108246186a48a379cdab28
2020-05-05 15:03:57 -07:00
Durham Goode
1cd598279e backout D21348756 about printing progress of downloads
Summary:
The formatter is broken, it prints to stdout which breaks json output,
and it's questionable whether we want a new line printed to the user for every
200k files downloaded.

Reviewed By: quark-zju

Differential Revision: D21409529

fbshipit-source-id: 2a5d94f8fcc62dbe25f0aae5453e4664c49fdcb2
2020-05-05 14:33:22 -07:00
svcscm
cdf03496b7 Updating submodules
Summary:
GitHub commits:

e5627af768
2f1700c8c5

Reviewed By: jurajh-fb

fbshipit-source-id: 894daf4bcb2939deb9709ba86887665a5e449a94
2020-05-05 14:33:22 -07:00
svcscm
70a5e0e788 Updating submodules
Summary:
GitHub commits:

6489eda488
633e1d7e3a
1797e36654

Reviewed By: jurajh-fb

fbshipit-source-id: d7ecf0c4f20e4dfdc9df3becf3a35c83757b58e4
2020-05-05 12:53:11 -07:00
Thomas Orozco
3b14b52337 mononoke/scuba_ext: allow truncating overly long msg fields
Summary:
When we log to Scuba, we need to truncate the `msg` field if it's too long, or
we might be missing log entries in Scuba. I put this behind a tunable so we can, well,
tune it.

Reviewed By: farnz

Differential Revision: D21405959

fbshipit-source-id: 08f0d3491d1a9728b0ca9221436dee2e8f1a17eb
2020-05-05 12:28:51 -07:00
svcscm
437d84726c Updating submodules
Summary:
GitHub commits:

3091f9dd0a
e5c563487b
449f3c089d
91bc0130fa
00e6af0887
42a643e797

Reviewed By: jurajh-fb

fbshipit-source-id: 5b656cd437a210119c7f5645f138971d96b3c542
2020-05-05 12:17:03 -07:00
Chad Austin
b9ba1b40af fix help text for info subcommand
Summary:
The name of a checkout is not a public concept, so refer to the
checkout's path.

Reviewed By: genevievehelsel

Differential Revision: D21393208

fbshipit-source-id: 4c014a6f65515f4632f2dffe5d563d0ee859dda0
2020-05-05 12:09:12 -07:00
Chad Austin
e0cfe9e104 rename stats io to stats fuse and stats latency to stats fuse-latency
Summary:
When I want FUSE stats, I never run `eden stats io`. To unify the
command language, rename `io` to `fuse` and `latency` to
`fuse-latency`, to match `thrift` and `thrift-latency`.

Leave aliases in place in the off chance someone cares.

Reviewed By: genevievehelsel

Differential Revision: D21392931

fbshipit-source-id: 843c1c85ea0aa162ba167f251f0f2cde5a389e72
2020-05-05 12:09:12 -07:00
Chad Austin
8b5c1e66b7 remove stats memory command
Summary:
`eden stats memory` was never useful because it shows system
statistics which you can get better from `top`, `htop`, or `atop`.

Reviewed By: genevievehelsel

Differential Revision: D21392737

fbshipit-source-id: 010021b8a97bd8ba8ac289d906acc3c3ecd10768
2020-05-05 12:09:11 -07:00
Zeyi (Rice) Fan
1c4c0f4024 fix perfect forwarding problem
Summary: So the linter will stop nagging at my changes to this file.

Reviewed By: wez

Differential Revision: D21396011

fbshipit-source-id: dbc3479637cca83aa1a11ff94a9033dfa42fc2a6
2020-05-05 11:46:21 -07:00
Zeyi (Rice) Fan
3baa8cc9b4 check if the blob fetching is present locally
Summary:
Talked with xavierd last week and we can use LocalStore's `get_missing` to determine if a blob is present locally. In this way we can prevent the backingstore crate from accidentally asking EdenAPI for a blob, so better control at EdenFS level.

With this change, we can use this function at the time where a blob import request is created with confidence that this should be short cheap call.

This diff should not change any behavior or performance.

Reviewed By: xavierd

Differential Revision: D21391959

fbshipit-source-id: fd31687da1e048262cb4eae2974cab6d8915a76d
2020-05-05 11:14:40 -07:00
Thomas Orozco
cef5d8d956 mononoke: test-linknodes: add more testing
Summary:
This test is flaky right now, but it's not clear why. I'm also unable to repro.
Let's add more logging.

Reviewed By: StanislavGlebik

Differential Revision: D21405284

fbshipit-source-id: 3ce5768066091de61e62339286410a6223d251d5
2020-05-05 10:32:55 -07:00
Durham Goode
1da0256921 discovery: use the latest commits for discovery instead of random ones
Summary:
When doing discovery, for repos with long master lines and infrequent
branches, picking a random set of sample commits could result in not picking the
master, and therefore having to do very long commit graph traversals to check
ancestors against the other samples.

To prevent this, let's pick the N most recent commits instead of a random
sample. This should generally get the master commit into our sample.

Reviewed By: quark-zju

Differential Revision: D21394302

fbshipit-source-id: f4b8110cd126b90553ec624e48cab0b590e124fb
2020-05-05 09:56:26 -07:00
Lukas Piatkowski
621678a798 mononoke/time_window_counter: use trait objects to represent time_window_counter
Summary: Making a trait out of TimeWindowCounter will help with providing different implementations of load limiting for OSS and FB.

Reviewed By: krallin

Differential Revision: D21329265

fbshipit-source-id: 7f317f8e9118493f3dcbadb0519eaff565cbd882
2020-05-05 08:02:50 -07:00
svcscm
3b8c8b4680 Updating submodules
Summary:
GitHub commits:

1972a4b5c2
1cd11098a9

Reviewed By: jurajh-fb

fbshipit-source-id: 5ffb2e9251741c8db314dd2038796fc5d9abb951
2020-05-05 08:02:49 -07:00
svcscm
69006edc25 Updating submodules
Summary:
GitHub commits:

886a2a4651
313dba04bb
b809094278
14efec15bd
ca0252fe54
d25d21b400
f59693dbbc
61f194f57b
659027363b
9ebed36edf
dfdbb41163

Reviewed By: jurajh-fb

fbshipit-source-id: c5902f7e09d4106d16fb72e51b37099c2da192f9
2020-05-05 06:57:49 -07:00
Thomas Orozco
0dd85d1c15 mononoke/cache_warmup: add a bit more error context
Summary:
This is helpful. Also, while in there, I removed an error that wasn't used at
all.

Reviewed By: StanislavGlebik

Differential Revision: D21399489

fbshipit-source-id: 0e5ef20b842afa9ffc0bb8530c48eb48339c558e
2020-05-05 05:44:53 -07:00
Thomas Orozco
9ac8e0505b mononoke: update various error enums to use #[source]
Summary:
We have a number of error enums that wrap an existing errors, but fail to
register the underlying error as a `#[source]`. This results in truncated
context chains when we print the error. This fixes that. It also removes a
bunch of manual `From` implementation that can be provided by thiserror's
`#[from]`.

This also required updating the `Display` implementation for those errors. I've
opted for not printing the underlying error, since the context chain will
include it. This does mean that if we print one of those errors without the
context chain (i.e. `{}` as opposed to `{:#}` or `{:?}`), then we'll lose out a
bit of context. That said, this should be OK, as we really shouldn't ever being
do this, because we'd be missing the rest of the chain anyways.

Reviewed By: StanislavGlebik

Differential Revision: D21399490

fbshipit-source-id: a970a7ef0a9404e51ea3b59d783ceb7bf33f7328
2020-05-05 05:44:52 -07:00
Thomas Orozco
fd6b1d4ec6 common/rust/failure_ext: get rid of error chain
Summary:
This removes our own (Mononoke's) implementation of failure chains, and instead
replaces them with usage of Anyhow. This doesn't appear to be used anywhere
besides Mononoke.

The historical motivation for failure chains was to make context introspectable
back when we were using Failure. However, we're not using Failure anymore, and
Anyhow does that out of the box with its `context` method, which you can
downcast to the original error or any of the context instances:

https://docs.rs/anyhow/1.0.28/anyhow/trait.Context.html#effect-on-downcasting

Reviewed By: StanislavGlebik

Differential Revision: D21384015

fbshipit-source-id: 1dc08b4b38edf8f9a2c69a1e1572d385c7063dbe
2020-05-05 05:44:52 -07:00
Thomas Orozco
70007f049e mononoke/lfs_server: add a test for error formatting
Summary:
I'm going to send a diff to get rid of failure chains, and the LFS Server
actually uses that quite a bit. Let's make sure we don't affect the error
rendering there.

Reviewed By: StanislavGlebik

Differential Revision: D21383032

fbshipit-source-id: e0ec9c88760e7fd48d39fa1570efd1870a9ef532
2020-05-05 05:44:52 -07:00
svcscm
e130a8a8e3 Updating submodules
Summary:
GitHub commits:

79730fef2d
c5f5dc0071
2bf5443abd
7c383d4e61
d2b3dbf4f9
ce326f3523
a8053d8d0d
f2c6d7c349
f289daa8f3
43fdf5c767
fb0bffd5aa
8369f9d9b3

Reviewed By: jurajh-fb

fbshipit-source-id: 0885170ab34e4c0e096ba8f7b7078a05432faacb
2020-05-05 05:07:53 -07:00
Lukas Piatkowski
2da413da3e mononoke/opensource: temporarily remove mononoke manifest
Summary: The oss jobs are causing problems to the developers now, lets disable them temporarily.

Reviewed By: StanislavGlebik

Differential Revision: D21400898

fbshipit-source-id: f7a3567056633d9eef98a8d05a37cd029c9e506c
2020-05-05 04:20:42 -07:00
svcscm
659ecb4858 Updating submodules
Summary:
GitHub commits:

e2fba1fdf9

Reviewed By: jurajh-fb

fbshipit-source-id: 0cd1a9fad1773d95e69b53b6f2915f1bdb251bdc
2020-05-05 03:46:42 -07:00
Thomas Orozco
c63ac4a8eb mononoke: fix a broken test
Summary:
Looks like this broke yesterday. There was a Reindeer update yesterday IIRC, so
I'm guessing that's the cause. In any case, this is easy to fix forward.

Reviewed By: farnz

Differential Revision: D21399830

fbshipit-source-id: 5cf33411e089a8c675a8b3fdf7b6ae5ae267058d
2020-05-05 02:47:03 -07:00
svcscm
1d3f0f51b6 Updating submodules
Summary:
GitHub commits:

6fad3d5a80

Reviewed By: jurajh-fb

fbshipit-source-id: 630ed9b37a10e2a4f20f4b021d0c4c271fceb10a
2020-05-05 02:47:03 -07:00
svcscm
b9d6b776c0 Updating submodules
Summary:
GitHub commits:

2cabf118d2
cc70355a12
e53b3cc393
6f06ca6645
9cb9ae7ec8
d7f6c7d3eb
7d8f6d9e81
000f8ae8eb
a0bcb01daa
e7144be352
6ca7ea7e92

Reviewed By: jurajh-fb

fbshipit-source-id: 0e913373d6594492fbe249455754d36d70e3c9ab
2020-05-04 22:11:13 -07:00
Chad Austin
630ad5dbd6 default storage engine in config.toml
Summary:
If config.toml exists but does not contain a valid storage engine,
edenfs would fail to start. Now, it will set a default value and write
the updated config file.

Reviewed By: simpkins

Differential Revision: D19671453

fbshipit-source-id: 32f19dbed2be02bec2a96e1349dca6e7038b9301
2020-05-04 22:08:41 -07:00
Shannon Zhu
83dc7b9129 Custom pyre upgrade for eden
Summary:
Addressing issues simpkins brought up on D21207287 when we upgraded and introduced some pyre bugs.

Temporarily upgrading just this project, once we resolve some sandcastle capacity issues we'll release this via another global upgrade in fbcode.

Reviewed By: simpkins

Differential Revision: D21316793

fbshipit-source-id: f0c79f53d97f7182e7d8fe6e081c58ef53ce0c9a
2020-05-04 21:19:20 -07:00
svcscm
eb1e7f0597 Updating submodules
Summary:
GitHub commits:

2e48429390
faf91ce01f
89e187d839
257e071629
79952c99f2
5a61e7864d
eddb225388
93403f13b7
d21b67d26d
fedb9fe464
34b8b26990
26ffeccf3e
c0db94e58f

Reviewed By: jurajh-fb

fbshipit-source-id: 94972f903e6fe8e28f857572aceb001fb788eb80
2020-05-04 20:32:25 -07:00
Zeyi (Rice) Fan
da28f5a5b1 revisionstore: create directory with group share permission in correct places
Summary: When we create directory at certain places, we want these directories to be shared between different users on the same machine. This Diff uses the previously added `create_shared_dir` function to create these directories.

Reviewed By: xavierd

Differential Revision: D21322776

fbshipit-source-id: 5af01d0fc79c8d2bc5f946c105d74935ff92daf2
2020-05-04 19:21:33 -07:00