Commit Graph

62786 Commits

Author SHA1 Message Date
svcscm
ea10517cdf Updating submodules
Summary:
GitHub commits:

b058d43dcc
4b136c1a70
b23740a8c4
65c5515554
28a239d6de
e23e76de60
36963dc2ca
a024f312e0
b51baaeaf5
d3c684042e
07c486fe68
344d0df623
30e3b4c92e
c030e54a6f
1a47ad4882
bbb04cd6f8
deaaaa74c7

Reviewed By: wittgenst

fbshipit-source-id: 7cb1719a0baaaae724a92cc2073b73a96f762400
2021-01-27 11:07:01 -08:00
Johan Schuijt-Li
d26b084530 show certificate message on ssl errors
Summary:
When a cert is expired, it can happen that the SSL error isn't raised till
after all our TLS handling, but in performing the HTTP request. Check the
exception type to display our certificate warning in these cases as well.

Reviewed By: krallin

Differential Revision: D26101231

fbshipit-source-id: 8342db5908e1b5fac9973fe48de432d591e9f54e
2021-01-27 08:53:27 -08:00
Stanislau Hlebik
2623c27962 mononoke: use save_bonsai_changesets function
Summary:
Let's use this function since it removes copy-paste and adds additional checks
- for example it checks that parent are in blobstore.

Reviewed By: krallin

Differential Revision: D26079176

fbshipit-source-id: 9cd9bd170b929fa66c691432417a952ce11028ab
2021-01-27 08:45:44 -08:00
Thomas Orozco
2768bb08d2 mononoke: hg sync job: only sync globalrevs for the publishing bookmark
Summary:
We want multiple bookmarks, but only one of them should assign new globalrevs,
so it follows that we shouldn't sync the counter when other bookmarks are being
moved.

This does that.

Reviewed By: ahornby

Differential Revision: D26076567

fbshipit-source-id: 0ccc311984d3379cb44ccf10cbcb90ac31b82ee3
2021-01-27 08:32:39 -08:00
Thomas Orozco
2ca2e8b123 mononoke: read globalrevs enabled from globalrevs_publishing_bookmark
Summary:
Like it says in the title. This is prep work for allowing extra
bookmarks in a Gobalrevs repo later in this stack.

Reviewed By: ahornby

Differential Revision: D26076566

fbshipit-source-id: c775d50dfaa51e0f0f64e861b6c5b7ee16d62074
2021-01-27 08:32:38 -08:00
Thomas Orozco
d863816e0e mononoke: sync Configerator repos schema
Summary: See D26019054 for what this does.

Reviewed By: StanislavGlebik

Differential Revision: D26019125

fbshipit-source-id: 071f0115d5ee8435bbc7ff5208ace8ef6e948272
2021-01-27 08:32:38 -08:00
Shayne Fletcher
38db03ad7f Implement a Rust FFI for embedding bytecode generation in C++
Summary:
Bytecode generation entry point `compile_ffi::compile_from_text_cpp_ffi(...)` and a driver to exercise it:  `hh_single_compile_cpp`.

What we get here is a C FFI for generating HHAS from source text. We can tweak the interface as we need to going forward but this will do for a first version and I think has roughly feature parity with the OCaml FFI:
- `compile_ffi.rs` defines the function;
- `compile_ffi.h` defines the C interface;
- `hh_single_compile_cpp.cpp` provides a C++ CLI that calls the function.

Reviewed By: shiqicao

Differential Revision: D25979967

fbshipit-source-id: 4b46f9af23c61150dda6c33f9fa14e2c455c54c2
2021-01-27 08:08:38 -08:00
Stanislau Hlebik
5eedbf488b mononoke: dump pretty json when changing xrepo mapping
Summary: It makes it more readable

Reviewed By: ikostia

Differential Revision: D26100720

fbshipit-source-id: 7f9b37f1e3e9f4a476e89ff59740004070d066f9
2021-01-27 07:46:05 -08:00
Liubov Dmitrieva
339788a95e fix tests
Summary: Fix tests broken by the deprecation notice D26073439 (14a0d96b04)

Reviewed By: johansglock

Differential Revision: D26099594

fbshipit-source-id: 65b973c4f1dcc7a032e747c12ce59a1d0c04091e
2021-01-27 05:12:09 -08:00
Stanislau Hlebik
734928ecb9 mononoke: move functions from rsync admin to copy_utils
Summary:
I plan to use these functions in the megarepotool, so let's move them to a
library that can be used in both.

Reviewed By: krallin

Differential Revision: D26015773

fbshipit-source-id: 0d2d28d86471c417508494883b69fb64e1bea328
2021-01-27 02:47:04 -08:00
Evgenii Kazakov
e05104c8c2 mononoke/tunables: add by_repo tunables
Summary: Added new bool tunables type, depending on repos.

Reviewed By: ikostia

Differential Revision: D26025163

fbshipit-source-id: 5c2fa846b3c115bea683be249e92d495d28c038c
2021-01-27 01:50:05 -08:00
Alex Hornby
e3f3849d0b mononoke: add enum representing the types the walker interns
Summary: Add a new enum representing the types the walker interns so that we can optionally clear them between chunks. This change adds the enum and command line parsing, actual clearing follows in the next diff.

Reviewed By: krallin

Differential Revision: D25910622

fbshipit-source-id: 0226b4009bf8199498e21e52f734a9529ee7afaa
2021-01-27 01:15:24 -08:00
Neil Mitchell
ff01567b62 Add noexcept around ImportPriority
Summary:
With certain compiler flags:

```
#include <atomic>

struct ImportPriority {
  ImportPriority() {}
};

std::atomic<ImportPriority> f() {
  throw "";
}
```

Fails with with:

```
libgcc/include/c++/trunk/atomic:194:7: error: exception specification of explicitly defaulted default constructor does not match the calculated one
      atomic() noexcept = default;
      ^
Demo.cpp:15:29: note: in instantiation of template class 'std::atomic<ImportPriority>' requested here
std::atomic<ImportPriority> f() {
                            ^
1 error generated.
```

Reading the C++ spec, constructors default to noexcept, but it's not clear if that's true when they have a body. There are flags you can set that make this compile, but let's be maximally compatible and add `noexcept`.

Reviewed By: fanzeyi

Differential Revision: D26076451

fbshipit-source-id: 2f63256377fb31fd7867d7b03e7572e033f72dfc
2021-01-27 00:33:18 -08:00
Pedro Eugenio Rocha Pedreira
6223c3dc91 Add getdeps.py manifest
Summary: Adding simple getdeps.py manifest for f4d.

Reviewed By: amitkdutta

Differential Revision: D25842791

fbshipit-source-id: eabc33ec526c454f301f8cd401156f6250e5eaa1
2021-01-26 21:39:52 -08:00
Arun Kulshreshtha
9f7d4447fd auth: optionally validate client certs
Summary:
This diff revives D25454687 (f98273063a), which was backed out by D25792491 (b52168c4c8) because it was causing Mercurial to crash in certain environments where certificates are configured incorrectly.

I've modified the code so that by default, certificates are not validated (maintaining the old behavior), but users of the API can opt-in to validation. In the case of EdenAPI (which is the only user that opts in), this is controlled via a new `edenapi.validate-certs` config option, which defaults to false. This allows enforcing validation on platforms where the configs should be correct (such as devservers) while maintaining the old behavior on other platforms by default.

Reviewed By: DurhamG

Differential Revision: D26009207

fbshipit-source-id: 904dee61fd12fdee4a0031d14adef7fdb4801139
2021-01-26 17:58:12 -08:00
Jun Wu
ab03409db5 dag: make LazySet support async iterator (aka. Stream)
Summary:
Previously the LazySet only supports non-async Iterator. This makes it more
flexible useful. It will be used in upcoming changes.

Reviewed By: sfilipco

Differential Revision: D25858800

fbshipit-source-id: 8c8e874f05cfab721bc0fa55160a9337ed7c2c27
2021-01-26 17:11:09 -08:00
Jun Wu
84cd4fd250 dag: add indexedlog feature to gate out indexedlog
Summary:
In the future we'd like to allow building the dag crate without the indexedlog
portion.  This diff adds support for that.

Reviewed By: DurhamG

Differential Revision: D25769054

fbshipit-source-id: eb5a200841f878836a9f68e65e7d50be7e6b9a79
2021-01-26 17:11:08 -08:00
Jun Wu
53e6f4bac2 dag: move dag subset used by edenapi-types to a separate crate
Summary:
In the future we want to build dag without indexedlog dep for Mononoke
use-case. One of the problem is the ToWire trait implemented on dag::Id by
edenapi-types. Within buck, the dag crate will have 2 targets: dag and dag-lite
(no indexedlog). They are incompatible meaning that edenapi-types depending on
dag-lite will not provide Id::to_wire for crates using dag, or vice-versa.

To solve that, we move the Id and other types to a separate crate that only has
one buck target so edenapi-types, and segmented_changelog from Mononoke can
depend on it without issues. This also makes edenapi-types more lightweight.

Reviewed By: sfilipco

Differential Revision: D25857917

fbshipit-source-id: d3e15a2b6638cc6e15171a1e9bc37362e03df583
2021-01-26 17:11:08 -08:00
Jun Wu
a3f7dc77a8 dag: move id.not_found() to a trait
Summary: In upcoming changes, we're moving Id to a separate crate. This makes that easier.

Reviewed By: sfilipco

Differential Revision: D25857918

fbshipit-source-id: 6e2163f6fa171d4cd3be4fc0c4c248fd87ba739c
2021-01-26 17:11:08 -08:00
Chad Austin
e83011241d substitute surrogates in log file (#8076)
Summary:
Pull Request resolved: https://github.com/facebookincubator/resctl/pull/8076

D26025779 (e282452d65) may have broken the getdeps build when logging
surrogates. Use errors=surrogateescape to try to avoid that.

Reviewed By: danobi

Differential Revision: D26079717

fbshipit-source-id: 98d938bfced417e0b39a6dcddb241225ffb69c06
2021-01-26 14:41:39 -08:00
Chad Austin
939e23a0e1 fix spurious fbwhoami crashes on corp
Summary:
The FbWhoAmI background thread throws an exception when /etc/fbwhoami
cannot be opened, which terminates the process. This happens to people
in various spurious situations on corp. Since EdenFS doesn't care, let
FbWhoAmI return empty string values in that case.

Reviewed By: genevievehelsel

Differential Revision: D26010530

fbshipit-source-id: b0e9294e45f6fc2f7547982692de8ff09939405b
2021-01-26 13:20:09 -08:00
Xavier Deguillard
c198382acf nfs: add XDR serialization helpers
Summary:
External Data Representation (XDR) is an encoding scheme that NFS and all the
related protocols are using. As EdenFS will soon gain the ability to expose
itself via NFS, let's add some code to encode/decode data to XDR.

This code was written by wez.

Reviewed By: kmancini

Differential Revision: D25986692

fbshipit-source-id: ed4ce52ac87fc9396219b35c9e143e5471039fee
2021-01-26 12:35:03 -08:00
Alexey Spiridonov
3d80f505f6 Stop trying to build on Darwin, Windows
Reviewed By: nilesr

Differential Revision: D26059442

fbshipit-source-id: d323ab38d7171b344c5a15863ec8d0a70b877853
2021-01-26 11:16:07 -08:00
Mark Mendoza
64fdd43c5b adding hidden --preserve-mount-point flag to eden rm
Summary:
As described in D26033704, it would be really nice if we could have a version of `eden rm` that did everything except remove the mount point directory.  This creates a nice symmetry between the tolerated start and end points of a clone/rm cycle, in that `eden clone` already tolerates being targeted against and empty directory.

NOTE: I made no effort to make this work on Windows.  The implementation was different enough that I wasn't exactly sure how you all wanted to go about it.  We don't run on Windows at all, so this is all we need, but I'd understand if you all want to make it work, or at least throw an explanatory error.

Reviewed By: chadaustin

Differential Revision: D26033912

fbshipit-source-id: 36d63ad510393d3d09b1912a73efaebede534f61
2021-01-26 11:11:09 -08:00
Daniel Xu
47d073e81d Add fields to Cargo.toml's to make crates publishable
Summary:
This diff adds some basic information. Note I have no idea what most of
these crates do so the descriptions are a bit light. I'd appreciate it if people
could tell me what the descriptions should be. Or better yet, just put up a diff
for it after this one goes in.

Reviewed By: lukaspiatkowski

Differential Revision: D26031870

fbshipit-source-id: 496f44882b6ed80a4f3834cf0fa3215ab318c930
2021-01-26 10:58:14 -08:00
Stanislau Hlebik
e61938d63f mononoke: speed up verifying working copy equivalence by excluding paths
Summary:
For some large repo paths we can easily tell that they are don't need to be
visited e.g. if all small repo paths rewrite into a single directory in the
large repo.

This diff looks at CommitSyncConfig and limits what large repo paths need to be
visited.

Reviewed By: ikostia

Differential Revision: D26040696

fbshipit-source-id: db18c569c3a168f6defe5d38bb1aec95ec9623c1
2021-01-26 10:34:28 -08:00
Stanislau Hlebik
9e9446a972 mononoke: add an option to the command that changes xrepo mapping to also dump mapping to a file
Reviewed By: ikostia

Differential Revision: D26051581

fbshipit-source-id: 9d5ec2bcb5a14b8bf2ea0a3a8931f2bb70009daa
2021-01-26 08:59:51 -08:00
Liubov Dmitrieva
14a0d96b04 add deprecation notice for pullbackup related set of commands
Summary:
this is needed to remind users to use the commands related to the Commit Cloud Workspaces

This is also needed to avoid confusion - the current error is not very informative:

```
connected to twshared9522.24.frc3.facebook.com session Wfq9tGuoPSBJH8nc
abort: 'listkeyspatterns' command is not supported for the server ssh://hg.vip.facebook.com//data/scm/fbsource?infinitepush_write
```

Reviewed By: markbt

Differential Revision: D26073439

fbshipit-source-id: 58ee477bc021a0796e22cc18610225ee003e06be
2021-01-26 08:21:07 -08:00
Thomas Orozco
ad5dc2d59d third-party/rust: hyper-02: update to 0.13.9
Summary:
Just a minor version update. I'd like to add a patch on top of this for a PR
that hasn't been merged yet, but updating to the underlying released version
first will make the diff clearer.

Reviewed By: ahornby

Differential Revision: D26047997

fbshipit-source-id: 91856f645ec3aaaf4fbf256a23c7e8d4db0f6b37
2021-01-26 07:21:28 -08:00
Liubov Dmitrieva
2c15a29f0f fix unit test that was broken by D19803760
Summary: fix unit test that was broken by D19803760 (8e35bcc49d) to unblock release

Reviewed By: krallin

Differential Revision: D26073002

fbshipit-source-id: 88da60f5806ef5cb2f832922e503309836542e09
2021-01-26 04:33:47 -08:00
svcscm
432cef9395 Updating submodules
Summary:
GitHub commits:

abb2812be7

Reviewed By: wittgenst

fbshipit-source-id: 8084f2362decab720cd226e7bddfd5a0fe69d11d
2021-01-26 04:33:47 -08:00
svcscm
26ddb10827 Updating submodules
Summary:
GitHub commits:

91814ec04c
12f1137355
85165a158e

Reviewed By: wittgenst

fbshipit-source-id: ec0084104d834d837d221247167b04fc34b7a867
2021-01-25 22:58:27 -08:00
Daniel Xu
5715e58fce Add version specificiation to internal dependencies
Summary:
Lots of generated code in this diff. Only code change was in
`common/rust/cargo_from_buck/lib/cargo_generator.py`.

Path/git-only dependencies (ie `mydep = { path = "../foo/bar" }`) are not
publishable to crates.io. However, we are allowed to specify both a path/git
_and_ a version. When building locally, the path/git is chosen. When publishing,
the version on crates.io is chosen.

See https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#multiple-locations .

Note that I understand that not all autocargo projects are published on crates.io (yet).
The point of this diff is to allow projects to slowly start getting uploaded.
The end goal is autocargo generated `Cargo.toml`s that can be `cargo publish`ed
without further modification.

Reviewed By: lukaspiatkowski

Differential Revision: D26028982

fbshipit-source-id: f7b4c9d4f4dd004727202bd98ab10e201a21e88c
2021-01-25 22:10:24 -08:00
svcscm
014b7625da Updating submodules
Summary:
GitHub commits:

dfabe4ce46
1ab6a01f76

Reviewed By: wittgenst

fbshipit-source-id: 71ff6e03b4a63dfdeeab958ceba64088869461a9
2021-01-25 22:10:23 -08:00
svcscm
c2244fbbe0 Updating submodules
Summary:
GitHub commits:

a227d297d8
3c764a2d6c

Reviewed By: wittgenst

fbshipit-source-id: 5f424753571aa53f1125abcff6073d79023d1b27
2021-01-25 18:39:20 -08:00
svcscm
5ea945353b Updating submodules
Summary:
GitHub commits:

938cef3500
fe5710128d
b69bbdf0e2

Reviewed By: wittgenst

fbshipit-source-id: 503cd629478a0ace9379199ad2751454db225bb1
2021-01-25 18:39:20 -08:00
svcscm
e73984b49b Updating submodules
Summary:
GitHub commits:

c29293df0b
0fd2fc2fae
7c6611cbdb
1316e8c3bd

Reviewed By: wittgenst

fbshipit-source-id: 59088052e88d3d87392191b3e25189af65846a21
2021-01-25 16:38:51 -08:00
Genevieve Helsel
9db3bf1e25 run linter in eden/fs/py
Summary: just running the linter :)

Reviewed By: chadaustin

Differential Revision: D26000280

fbshipit-source-id: 23bc996ca361b208cd6f643e1440eb2a9aeec034
2021-01-25 16:13:55 -08:00
Genevieve Helsel
7565ff7f59 run linter in eden/fs/docs
Summary: Just running the linter :)

Reviewed By: singhsrb

Differential Revision: D26000274

fbshipit-source-id: 5d94abf11210fda5e956c408764aa0c348aa0d84
2021-01-25 16:13:54 -08:00
Genevieve Helsel
55da8ffcbc run linter in eden/integration
Summary: just running the linter!

Reviewed By: chadaustin

Differential Revision: D26000269

fbshipit-source-id: 184eb962fa7b88eb9b8b6bd22ae76477cbb6a06c
2021-01-25 16:13:54 -08:00
svcscm
ba6971a797 Updating submodules
Summary:
GitHub commits:

07a3a628d5
88382fbfe7
efadee9e20
1d226018af
ed1f0417c7

Reviewed By: wittgenst

fbshipit-source-id: 611077d2ea17b264ed9c147833ae2585f88d2e0b
2021-01-25 16:13:54 -08:00
svcscm
8b91f01b7f Updating submodules
Summary:
GitHub commits:

7b67e06762
3188750513
3657644b0f
88547cb221
431e8afba7
7f1185f4cd
8bb14ecd8d
757eff8093
33c87bb7b6
fd40801554
f243a25318
3e56221374

Reviewed By: wittgenst

fbshipit-source-id: e0b74e240f732addd4f1330560429195661cf536
2021-01-25 14:16:40 -08:00
Stanislau Hlebik
fa5b61f577 mononoke: add rpm for megarepotool
Summary: I'd like to install it on scm devservers, and that's the easiest way to do it.

Reviewed By: HarveyHunt

Differential Revision: D26048848

fbshipit-source-id: d463f99b710b60b34a06fb0bc8cbe2af9fe418f2
2021-01-25 13:30:08 -08:00
svcscm
4d166211c7 Updating submodules
Summary:
GitHub commits:

2d173fe622
f906ccd95d
b1e069e477
4731bb0c37
2b96889a0f
55c183ba85
ad2415d3b7
ed5fa7611b
d43298f2cb
767af9fad6
5f6279b2a7
57bfe6497c
5ab476fe0c

Reviewed By: wittgenst

fbshipit-source-id: f62aeff0d066927b967ac855e469042c4670ed17
2021-01-25 13:17:10 -08:00
Chengxiong Ruan
85e4390166 Fix weird looking highlighted values in below (#8074)
Summary:
Pull Request resolved: https://github.com/facebookincubator/resctl/pull/8074

1. Upgrade `Cursive` to v0.16.3
2. patch `cursive_buffered_backend` with a temp fork before v0.4.2 is released. This is needed to avoid Rust being confused by `Backend` trait from different versions of `cursive_core`

Reviewed By: dschatzberg

Differential Revision: D25987535

fbshipit-source-id: 6826653f9b6d4d6bdba485abf42eb517a853acc1
2021-01-25 13:12:37 -08:00
Chad Austin
4196d44f5e include FUSE result codes in eden strace
Summary: Like `strace`, show the return codes from FUSE requests.

Reviewed By: kmancini

Differential Revision: D26033195

fbshipit-source-id: 2347129ce480e50a3b0f588937e535e0df45dfbd
2021-01-25 12:35:36 -08:00
Chad Austin
73b0d2a4a4 log the name and mode for FUSE_CREATE requests
Summary:
Correct an oversight that did not show the name of the created file in
a FUSE_CREATE request.

Reviewed By: kmancini

Differential Revision: D26033166

fbshipit-source-id: 35eb8a844aff29b519318109c918f2a700692835
2021-01-25 12:35:36 -08:00
Chad Austin
e282452d65 always write log files as UTF-8 to avoid cp1252 encoding errors
Summary:
The log file should be opened with a defined encoding rather than
whatever the platform default is. On Windows, that might be cp1252,
which throws an encoding error when the text is unicode.

Reviewed By: genevievehelsel

Differential Revision: D26025779

fbshipit-source-id: 0102b2555e91812dfe94a7c332eb896ca2c9ef7e
2021-01-25 12:35:36 -08:00
svcscm
38dc12d95b Updating submodules
Summary:
GitHub commits:

bf66d6275b
f313b0238d

Reviewed By: wittgenst

fbshipit-source-id: f9353dd1aa1898cf5af124aef6557b2d11a63608
2021-01-25 11:00:28 -08:00
Alex Hornby
217c07c65a mononoke: introduce tail params to walker
Summary:
We are going to pass more params into tail.rs as part of being able to clear state between chunks.

This prepares by creating a struct for them and adding command line args for include/exclude node types to clear.

Reviewed By: krallin

Differential Revision: D25910615

fbshipit-source-id: 610a884c17da7af1e23cfa81d4f495fe03bad9a3
2021-01-25 09:11:07 -08:00