Summary:
This is a pretty in depth third party library being added. The main changes are...
**version bumping**:
dunce = "1.0.0" => "1.0.2"
once_cell = "1.8" => "1.12"
tracing = "0.1.32" => "0.1.35"
adding ethers-rs fork:
https://github.com/rlkelly/ethers-rs.git
this depends on forks of:
coins-bip32 = https://github.com/rlkelly/bitcoins-rs.git
coins-bip39 = https://github.com/rlkelly/bitcoins-rs.git
eth-keystore = https://github.com/rlkelly/eth-keystore-rs.git
this was necessary to remove the wasm target, which creates a cyclic dependency for "indexmap". This was ran into previously here:
https://fb.workplace.com/groups/rust.language/permalink/8603206789727860/
also, some fixups were created for a few packages to facilitate build. I'm not a fan of libraries creating a build step to generate documentation...
----
This still does not have the ethers-middleware crate or the ethers-solc crate because it has the external build dependency of solc, which would make this a bit more complex to add:
https://docs.soliditylang.org/en/latest/installing-solidity.html
adding the crates without solc makes the process of integrating this library much easier, but it would be beneficial in the future.
@public
update rust deps
Reviewed By: jsgf
Differential Revision: D37320246
fbshipit-source-id: 251bd1c3c0e51733347a534fa597dea33d1df74b
Summary:
When given a (commit, path) pair, `AclRegions` facet returns all Hipster ACLs that permit accessing that (commit, path). See D34272143 for how configuration works.
To answer those queries effectively, we precompute a trie over all path prefixes from rules in the config. Then, for each (commit, path) query:
1. Use the trie to extract all rules that match by path.
2. For all rules we got from #1 naively check that the given commit matches them.
Design doc: https://docs.google.com/document/d/1MQBk-N-kRS7L-7IoFzkN0BdCif1UfyDQd0P1qOnx8TQ/edit?usp=sharing
Reviewed By: markbt
Differential Revision: D34674134
fbshipit-source-id: b7ca5c6a1651cd6ef379e5a4d12068f15492c496
Summary: Segmented Changelog requires `CoreContext` for construction, so we can create a test mock if `TestRepoFactory` has a `FacebookInit`.
Reviewed By: markbt, mitrandir77
Differential Revision: D35251021
fbshipit-source-id: 57051e227804669794f19fc2ca130f52cb938b0c
Summary:
Having each repo fixture as a separate module with repetitive code looks wrong. The repetitive code is also inconsistent between different fixtures.
I introduced a proper trait with default function implementations and replaced each module with a struct. The end code is shorter and cleaner, but I needed to modify a lot of other files because of naming changes and `use fixtures::TestRepoFixture`.
Reviewed By: markbt
Differential Revision: D35081002
fbshipit-source-id: fa753d933665afcd89cd902b20205de1968de29c
Summary:
Replace all uses of `BlobRepoHg::get_hg_from_bonsai_changeset` with
`DeriveHgChangeset::derive_hg_changeset`. These are equivalent, except that
the latter does not depend on `BlobRepo`.
Reviewed By: mitrandir77
Differential Revision: D34971451
fbshipit-source-id: 4c346e9e6f102759a54c9d0cebc6c609b0843581
Summary:
To allow tests that use the `tests_utils` crate to work with repos other than
`BlobRepo`, migrate this crate to using facet reference definitions.
In a lot of tests, this was the only thing that required `BlobRepo`, which
means type inferencing now does not know which concrete type to use. In these
cases, we specify `BlobRepo` for now. In the future these can be changed to a
simpler repo type for that test.
Reviewed By: yancouto
Differential Revision: D34864664
fbshipit-source-id: e4142e8e47c3e942a5d9ceac4ba4270fe597f097
Summary:
This diff does the following:
- Updates the tracing crates to the latest versions available. Changes include:
- Experimental support for [Valuable](https://tokio.rs/blog/2021-05-valuable), which makes recording values much more like how serde does. Valuable also makes it easier to write a tracing-slog crate due to some implementation details of slog where buffering of values is necessary.
- Combinators for combining filters in tracing-subscriber.
- A `MakeWriter` implementation for `RollingFileAppender` in `tracing_appender`, which allows the `RollingFileAppender` without a background thread to being spun up.
- Reduced the disabled span drop overhead by 50-70%—this might be most apparent in tests that don't set a subscriber.
- Move the tokio-console off git dependencies and to published versions on crates.io.
This diff also updates the following, non-exhaustive list of crates:
- parking_lot
- prost
- thread_local
- tonic
- hyper
- httparse
- h2
allow-large-files
Reviewed By: jsgf, fanzeyi
Differential Revision: D34801437
fbshipit-source-id: 2409de9194c8e00647ac5e4c514325fd2e23d247
Summary:
Use the `ArcChangesetFetcher` type (which is `Arc<dyn ChangesetFetcher + Send + Sync>`)
so that it is compatible with the facet type.
Reviewed By: Croohand
Differential Revision: D34203295
fbshipit-source-id: 6be4d6014bde8fb5461c9ce775b597ae2184b31d
Summary:
There are currently multiple ways to map between bonsai and hg changesets: using the bonsai-hg mapping repo attribute, or using the method on blobrepo that is very similar (but takes `ctx` by value). Standardize on using the repo attribute, and convert all callers.
There is one exception to this: when callers wish to trigger derivation, they
continue to call `BlobRepoHg::get_hg_bonsai_mapping`. Despite its name, this
method triggers derivation in the bonsai-to-hg direction only. We will
separate out this functionality in the future.
Reviewed By: mitrandir77
Differential Revision: D34071889
fbshipit-source-id: f0c4ce7987376702dc36eb27683d1fcb8a22df60
Summary:
The published `terminfo` crate has a WTFPL license which is forbidden from being used by Meta.
{F687594009}
However the WTFPL allows forking and relicensing, so I've gone ahead and relicensed the same code as dual MIT OR Apache 2.0 for fbsource to use. We can patch the properly licensed code in place of all transitive dependencies on the `terminfo` crate, as they have the same public Rust API.
The relevant changes: 9b4e1c6aae
Reviewed By: metajack
Differential Revision: D33190944
fbshipit-source-id: 0a9002b8799b5ceeaca3c20a211f15beec5a407f
Summary: Run autocargo before doing any other changes, likely some or all of this will rebase away
Reviewed By: c-ryan747
Differential Revision: D33790131
fbshipit-source-id: fe6c761a09b300e482b5eb9c9c1465716d35dded
Summary:
You can't run Autocargo right now, it fails with:
```
Error: File CargoTomlPath { file: PathInFbcode("configerator/structs/data_access_policies/upf/eval/Cargo.toml"), dir: PathInFbcode("configerator/structs/data_access_policies/upf/eval") } is covered by both ctp and pxl projects
```
This is affecting me (on Buck2) since some of our dependencies use Autocargo
for their Cargo.tomls and those are now missing some dependencies (notably,
`below` needs `nix`). This also affects people trying to vendor crates:
https://fb.workplace.com/groups/rust.language/permalink/7625318880849994/https://fb.workplace.com/groups/rust.language/permalink/7577471608968055/
This diff fixes it, and regenerates Cargo.tomls as a result.
Note: I'm not sure exactly what "pxl" is, but it seems to be owned by the team working on data access so that's a better home for that crate than ctp.
Reviewed By: ahornby
Differential Revision: D33428045
fbshipit-source-id: f3feab3ae04069672040423c145c69a58445ef96
Summary: 0.27 onwards has support for large tuples so can remove patch
Reviewed By: farnz
Differential Revision: D33100620
fbshipit-source-id: 68348b16113043826dfb253acc64048d01bba7d1
Summary: The change is in upstream slog-syslog release 0.13.0 so we can remove the patch
Reviewed By: jsgf
Differential Revision: D33099611
fbshipit-source-id: a30b89179d18cbd7b3d9dbe6ac8a461b143b01c0
Summary: Last users are gone, lets remove these old crates
Reviewed By: wqfish
Differential Revision: D33061743
fbshipit-source-id: bea14f6f199a3f805d76318d3c8070739abc6085
Summary:
For invocations of `ensure!` where the argument is one of the comparison operators `==` `<=` `<` `!=` `>=` `>` and the lhs and rhs of the operator both have a `Debug` impl, we now render both sides as part of the error message.
### Example:
```
ensure!(flags.len() <= 40);
```
```
ensure!(kind == Kind::File);
```
Before:
```
Condition failed: `flags.len() <= 40`
Condition failed: `kind == Kind::File`
```
After:
```
Condition failed: `flags.len() <= 40` (99 vs 40)
Condition failed: `kind == Kind::File` (Symlink vs File)
```
Reviewed By: zertosh
Differential Revision: D32598267
fbshipit-source-id: 70fdab6d8f0e3ceceb98e651eab0a73795cadaeb
Summary:
This release helps catch places that the `anyhow!` or `bail!` macros are invoked with a missing fmt argument, such as:
```
- anyhow!("unsupported redirect operation {:?}")
+ anyhow!("unsupported redirect operation {:?}", op)
```
All instances of this in fbcode were fixed in D32266567 (2f440eac7e).
Reviewed By: wqfish
Differential Revision: D32564287
fbshipit-source-id: e9cf5a1376e6c83343de3f3c6ba2925d88334a30
Summary:
I am working on getting the `anyhow` macros to catch these bugs.
```
- anyhow!("unsupported redirect operation {:?}")
+ anyhow!("unsupported redirect operation {:?}", op)
```
Reviewed By: yancouto
Differential Revision: D32266567
fbshipit-source-id: b17efafac154d93da0c3ed70dea326156367a0d5
Summary: Bring the config in line with the oss branch names
Reviewed By: mzr
Differential Revision: D32109171
fbshipit-source-id: f87a0070b0ef06a49c86cacdfdc7f5c8b02f3fbf
Summary:
The 2021 edition has some new traits in the prelude, which means they no longer require an explicit import in every module that wants to use them.
Details: https://doc.rust-lang.org/edition-guide/rust-2021/prelude.html
Reviewed By: metajack
Differential Revision: D31823254
fbshipit-source-id: 7532a198347b032f0cf9277c916d54db268581ba
Summary: We need the rayon feature in the next diff in the stack
Reviewed By: alunyov
Differential Revision: D31833982
fbshipit-source-id: e908b3a8f8a16647df0bffe7a44115393254f7d9
Summary: Generated by `sed -i 's/edition = "2018"/edition = "2021"/' common/rust/cargo_from_buck/project_configs/*.toml` and `cargo autocargo` following the autocargo code change in D31830985 and its corresponding MSDK rebuild in D31832311.
Reviewed By: zertosh
Differential Revision: D31832698
fbshipit-source-id: db79024cad311467d5b7c13b7abedc6464f09366
Summary: This crate is a dependency of autocargo. The update is necessary in order for autocargo to be able to emit 2021 edition Cargo.tomls.
Reviewed By: zertosh
Differential Revision: D31824537
fbshipit-source-id: 1261e34a2f49d407afbb55a39acb73a4b4b237f6
Summary:
Mononoke's internal thrift types (e.g. BonsaiChangeset) should opt-in to
the `rust.exhaustive` annotation. This will mean we can statically check
that all uses of these types handle any additions we make in the future.
Reviewed By: farnz
Differential Revision: D31826594
fbshipit-source-id: ec48b4b661abf360769ea4e03eb54cd7191f8510
Summary: Codemod to add `..Default::default()` to Rust thrift struct instantiations. This will be required by D31751301.
Reviewed By: dtolnay
Differential Revision: D31749612
fbshipit-source-id: 56b0081550d4a24a1741407c47ac342e6fec87f0
Summary: Bump in preparation for a new release
Reviewed By: brianc118
Differential Revision: D31657479
fbshipit-source-id: eeb2a08c54f6db75b6e515ca0966637f31b906a8
Summary:
Vendoring this patch: https://github.com/curl/curl/pull/7737 to curl-sys rust crate. On windows the hg client is using curl that's bundled with sys-curl. I need this patch to have unix domain sockets working in hg client on windows.
I had to manually vendor https://raw.githubusercontent.com/mzr/curl/57e7ec4dbe4dd2831de51f2644879387d2ea7b44/docs/INSTALL because reindeer didn't do it. IDK why.
oss-eden-{darwin,linux,windows}-getdeps fail with:
```
FAILED: eden/scm/lib/backingstore/CMakeFiles/rust_backingstore.cargo eden/scm/lib/backingstore/debug/libbackingstore.a eden/scm/lib/backingstore/release/libbackingstore.a
cd /data/sandcastle/temp/fbcode_builder_getdeps/shipit/eden/eden/scm/lib/backingstore && /data/sandcastle/temp/fbcode_builder_getdeps/installed/cmake-hQhVzQT-WzFKTeqXjLxo5lLi8IG4_MjX2-YRqptCUVs/bin/cmake -E remove -f /data/sandcastle/temp/fbcode_builder_getdeps/shipit/eden/eden/scm/lib/backingstore/Cargo.lock && /data/sandcastle/temp/fbcode_builder_getdeps/installed/cmake-hQhVzQT-WzFKTeqXjLxo5lLi8IG4_MjX2-YRqptCUVs/bin/cmake -E env CARGO_TARGET_DIR=/data/sandcastle/temp/fbcode_builder_getdeps/build/eden/eden/scm/lib/backingstore CARGO_HOME=/data/sandcastle/temp/fbcode_builder_getdeps/build/eden/_cargo_home cargo build --release -p backingstore --features fb
Blocking waiting for file lock on package cache
Blocking waiting for file lock on package cache
error: failed to calculate checksum of: /data/sandcastle/boxes/eden-trunk-hg-fbcode-fbsource/third-party/rust/vendor/curl-sys-0.4.45+curl-7.78.0/curl/docs/INSTALL
Caused by:
failed to open file `/data/sandcastle/boxes/eden-trunk-hg-fbcode-fbsource/third-party/rust/vendor/curl-sys-0.4.45+curl-7.78.0/curl/docs/INSTALL`
Caused by:
No such file or directory (os error 2)
```
Not idea how to fix this. Seems related to the fact that reindeer didn't vendor docs/INSTALL.
# EDIT:
# It might been caused by some bug in hg. now it's fine
# The failure in oss-eden-linux-getdeps looks unrelated (something with rocksdb)
Reviewed By: krallin
Differential Revision: D31370778
fbshipit-source-id: a1245f8cb6b58f5765e34c95dfd78325a8e6e457
Summary:
This diff adds snapshot tests for most eden api wire types, while at the same time making the testing code much smaller, including tests for "wire" and "serialize" roundtrips.
## Context:
The previous diff had added an easy way to add snapshot tests. This stack aims to simplify the wire protocol code needed to create/modify an endpoint. A good thing to do before that is to add snapshot tests to all wire types, so that if we change them in a refactor, we're confident they still work exactly the same. This will also be useful when a type is changed in the future.
## How this makes tests easier
- In order to create snapshot tests, we need example objects to test with. Luckily we already use a framework for generating example objects (quickcheck::Arbitrary), so the idea here is to use that to make snapshot tests as automatic as possible.
- At the same time, the "wire" and "serialize" roundtrip tests (which also used Arbitrary), can also be made more automatic.
Now, using a simple helper, `auto_wire_tests!(WireObjectName)`, it is possible to derive all three types of tests automatically. This makes the current code smaller, and safer as we now have the additional safety provided by snapshot tests.
## Observations
- Not all wire types had tests implemented for them (I assume because it was too much work doing so, and might have done that myself in the past), I only moved the ones that already had. I'll do another pass and add remaining objects on a following diff.
- There are a couple actual non-refactor changes. I'll add comments explaining those.
- quickcheck crate is using quite an old version. I tried updating but it snowballed into something much more complicated, so I kept using the old version. We'll need to get to it at some point, though.
Reviewed By: markbt
Differential Revision: D31019233
fbshipit-source-id: 30c4a90848d0a5dcaffb89b9a0cd1cebfe4ace55
Summary: Need this for cargo check/rust-analyzer to work.
Reviewed By: guswynn
Differential Revision: D31319911
fbshipit-source-id: ebd3fa72d8fc3667391a2067f95cab9e5f53301f
Summary:
No need to keep the Bytes live after compact_protocol::deserialize, can move them into it instead.
Makes it clearer bytes aren't reused, and should have some small effect on peak memory usage during deserialization by freeing the Bytes buffer earlier
Reviewed By: StanislavGlebik
Differential Revision: D31149815
fbshipit-source-id: 858914d2d8e3d91b5e863053dfeeb5d5ec37b9eb
Summary: ACE will have to run in situations where Chef has not run, but we'll need to be able to reliably write to the auth logs so Blackbird can properly build detections. So we need these crates so we can build the somewhat foolproof solution to ensure ACE logs all executions.
Reviewed By: farnz
Differential Revision: D31066559
fbshipit-source-id: 9fa3b5778cd2602bdeaac90a9daa758b117babfe
