Summary:
Autocargo V2 will use a more structured format for autocargo field
with the help of `cargo_toml` crate it will be easy to deserialize and handle
it.
Also the "include" field is apparently obsolete as it is used for cargo-publish (see https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields). From what I know this might be often wrong, especially if someone tries to publish a package from fbcode, then the private facebook folders might be shipped. Lets just not set it and in the new system one will be able to set it explicitly via autocargo parameter on a rule.
Reviewed By: ahornby
Differential Revision: D26339606
fbshipit-source-id: 510a01a4dd80b3efe58a14553b752009d516d651
Summary:
Make the struct and method names in this crate more clearly reflective of what they do:
- `Auth` -> `AuthGroup`
- `Auth::try_from` -> `AuthGroup::new`
- `AuthConfig` -> `AuthSection`
- `AuthConfig::new` -> `AuthSection::from_config`
- `AuthConfig::auth_for_url` -> `AuthSection::best_match_for`
Reviewed By: singhsrb
Differential Revision: D26436095
fbshipit-source-id: a5ec5d9c48d3b75a0ee166b74e5340f9c529eeae
Summary: This implements the same logic as the `edenfsctl uptime`.
Reviewed By: fanzeyi
Differential Revision: D26412789
fbshipit-source-id: ebcf5f0b4767025ea210f7e9c69116b79436d5d0
Summary:
In a non-segmented-chanelog repo, `last`, the revset function has suboptimal
path that makes the following query:
last(::b8f917d4c & public())
take a long time. Use `max` to fix it.
Reviewed By: singhsrb
Differential Revision: D26435439
fbshipit-source-id: dc854b4c6978d2925cab2cb8f217365ed9c7f146
Summary: The local patch has been upstreamed so switching back to upstream's new version of sshcerts with proper support for our SSH keys and certs
Reviewed By: Imxset21
Differential Revision: D26419743
fbshipit-source-id: c6e6d2a8f77623360636a68febebfc85ad6f5c26
Summary: Make `AuthConfig::auth_for_url` match the behavior of the Python `httpconnection.readauthforuri` function when an auth group has a URL prefix with a scheme. Previously, the Rust version did not allow schemes to be specified on the prefix, and instead required them to be specified in the "schemes" fields. The Python code allows a scheme in the prefix, which overrides the contents of the "schemes" field for that auth group if present.
Reviewed By: DurhamG
Differential Revision: D26419721
fbshipit-source-id: 909b52a2e37e2fc908d5eb56740dd41dda826033
Summary:
In Python, no passed in timeout means a 3s connection timeout, let's do the
same in Rust.
Reviewed By: chadaustin
Differential Revision: D26407991
fbshipit-source-id: ad2919e2cb72e5a113499d7e036ae285ecf9ae34
Summary: You can use `instance.get_config()` to get access to global EdenFS configurations
Reviewed By: chadaustin
Differential Revision: D26407350
fbshipit-source-id: 022cc59fd86b2711c15cfd781872465c6ada9081
Summary:
This diff adds `edenfs-config` for loading EdenFS configurations from various
locations.
Reviewed By: xavierd
Differential Revision: D26391272
fbshipit-source-id: 3d34da98b2d530e13cdd831d3dc204e44304c486
Summary:
**Problem:** EdenFS has the classic hierarchical configuration design. We load from `/etc/eden/edenfs.rc` first, then `/etc/eden/conf.d/*` then `$HOME/.edenrc`. The latter can overwrite the former. At the end we have a complete view of the configurations.
`serde` is great, but it can't give us the information of whether it generated a field from `serde(default)` or from de-serialization. So we can't just deserialize then merge the configuration files. We need that information, and nor serde should give us that functionality.
`stack-config` is created to load configurations with serde. It automatically generates code with the intermediate data structure and taking care of merging of multiple configuration files. It derives a data type based on the original struct but wrap each field with `Option<T>` and mark it with Serde. Then it generates the code to merge and build the final concrete configuration data type.
It does not care what kind of data format the configuration file is, as long as it can deserialize into the generate optional type, it accepts it.
Example, say I have a file with this structure:
```
#[derive(StackConfig)]
struct Config {
path: String,
}
```
Then this crate will generate:
```
mod __stack_config_private {
#[derive(serde::Deserialize)]
pub(super) struct ConfigOpt {
path: Option<String>
}
...
}
struct ConfigLoader {
layers: Vec<__stack_config_private::ConfigOpt>,
}
impl ConfigLoader {
fn new() { ... }
fn load(&mut self, layer: __stack_config_private::ConfigOpt> { ... }
fn build(self) -> Config { ... }
}
```
See `examples/parse.rs` for usage.
Reviewed By: xavierd
Differential Revision: D26377547
fbshipit-source-id: 1e07d9867742913fd76ed4f765160f0035a2f2a3
Summary:
Move these conversion related function and trait out of `hg` module so EdenFS can use it too. Changes:
* Moved `get_opt`, `get_or` and `get_or_default` directly into `ConfigSet`.
* Moved `FromConfigValue` and `ByteCount` into `configparser::convert`.
Reviewed By: quark-zju
Differential Revision: D26355403
fbshipit-source-id: 9096b7b737bc4a0cccee1a3883e89a323f864fac
Summary: This diff sets up debug logging for EdenFS CLI with tracing.
Reviewed By: chadaustin
Differential Revision: D26354205
fbshipit-source-id: bcc89fe3eaf4c7ae7642b8c20fd74fd3ea6dd4ee
Summary:
The push-pull test was broken by my recent change to make indexedlog
the default. It wasn't caught because the test is disabled in Python 3. It now
passes with Python 3 so let's enable it.
The repack-remove fix is just some flakiness I've seen lately.
Reviewed By: singhsrb
Differential Revision: D26412998
fbshipit-source-id: be4b648f31bd6dfbf6a6e5d2e382acb084461974
Summary:
The walker mostly checks for duplicates before emitting a new edge, at the same time recording the edge as visited to prevent duplicate edges.
However for derived data where the node may or may not be present, the node isn't considered visited until the node data is successfully loaded and seen in state.rs record_resolved_visit(). This leaves a gap where we could be executing multiple copies of the same node.
Reviewed By: farnz
Differential Revision: D26319139
fbshipit-source-id: 52ce28f15341f132d94ebc1ff5e8ee2f0dc2564a
Summary:
Sometimes many nodes will unfold with a common child, in which case it is desirable that they aren't scheduled at the same time.
This adds bounded_traversal_unique as a new variant to prevent those duplicates.
Doing the duplicate detection inside bounded_traversal_unique means we only need to keep scheduled_max copies of keys and can keep it in a regular HashMap, vs doing it at bounded_traversal_stream call sites which would mean keeping a copy for everything in the unscheduled queue and using DashMap or a lock around HashMap.
Reviewed By: farnz
Differential Revision: D26319137
fbshipit-source-id: 3174ce9e7db4edeb107d26f72575de886e6b2e39
Summary: Add Python bindings to the Rust `auth` crate, with the intention of replacing `httpconnection.readauthforuri`.
Reviewed By: quark-zju
Differential Revision: D26419447
fbshipit-source-id: dd13bea74961137790beb8c96120ebef99e3c313
Summary:
For some repositories (e.g. configerator-sitevars) running "git gc" is too
slow. At the same time after initial rsync it would be good to run at least
some kind of validation to check that repo is not completely corrupt.
This diff adds that. This code checks that HEAD and 100 of its ancestors have
the object that they reference are reachable.
Reviewed By: krallin
Differential Revision: D26423769
fbshipit-source-id: a3fcd9fc5c30e5bf0fdc1cd0fb9e03bdc2e1371d
Summary:
johansglock pointed out that Hyper is affected by CVE-2021-21299. Let's update
to a fixed version.
Reviewed By: farnz
Differential Revision: D26313854
fbshipit-source-id: 4db04d3044fb9f22a037bda0a88a5314f62f9dfc
Summary:
See the earlier diff for what flag controls.
When booting SCS, we poll a lot of nested FuturesUnordered. This results in
very inefficient behavior in Tokio's cooperative scheduling, and results in us
spending 50% of total our CPU (in fact, a full thread) on just yielding, with
most other threads being idle.
With this change, we use 20+ threads running work that is scheduled by the main
thread, which is what we want.
Note that this applies to all Mononoke binaries. This has only been especially
bad in SCS startup, but we've also not benefited from this feature anywhere,
so rather than leaving this footgun in other apps, let's take it out
everywhere.
Reviewed By: markbt, StanislavGlebik
Differential Revision: D26399889
fbshipit-source-id: 0a13e1275d367e49c2342cb85cb6cd0047cda224
Summary: Instead of doing per-repo rate-limiting checks, let's do total ones. All of the business logic stays the same, with the exception of a different counter used.
Reviewed By: farnz
Differential Revision: D26374353
fbshipit-source-id: 92006cd3e5dd194ac9e6531cbb19289fa73a63d2
Summary: This imports D26374072. See that diff for details.
Reviewed By: farnz
Differential Revision: D26374352
fbshipit-source-id: 71f39b5c606915bfbf421b366812fd40ebb6b4f9
Summary: The Python `readauthforuri` method will include *any* fields specified in the `[auth]` section for a given auth group, even if those fields aren't one of the expect ones (such as `cert`, `key`, etc). This is sometimes used in tests to attach additional information to an auth group. To support this in Rust, let's just collect all unknown fields into a `HashMap`.
Reviewed By: quark-zju
Differential Revision: D26416086
fbshipit-source-id: 0252e340e38850a54e24d54810e9abd77c566f63
Summary:
The auth crate is now able to check the presence and expiration of client certificates (D26009207 (9f7d4447fd)). When a problem is detected, it emits an `X509Error`, which specifies exactly what the problem is. Since this error always indicates a certificate issue, we can print out the message configured in `help.tlsauthhelp` (which is more specific than `help.tlshelp` from the previous diff).
Previously, Mercurial would attempt to use the certificate anyway, resulting in a difficult to understand error message. Although the previous diffs in this stack improved the error messages on any TLS failure, the `X509Error` messages are even more helpful.
Users can opt in to this certificate validation with `edenapi.validate-certs`. The functionality is gated on a config option to prevent Mercurial from crashing if certificates are misconfigured, but EdenAPI isn't being used.
Reviewed By: quark-zju
Differential Revision: D26385843
fbshipit-source-id: 9809f612f8aab3f2dd442d6dd8dc348f1af45296
Summary: Print out the help text configured in `help.tlshelp` upon hitting a `TlsError`. Note that in this case, we use `help.tlshelp` rather than `help.tlsauthhelp` since all we know in this case is that *some* kind of TLS error occurred.
Reviewed By: quark-zju
Differential Revision: D26385844
fbshipit-source-id: 1fb5280195de75107ecdfc9203ef8ddda2a04052
Summary: Add a new `TlsError` Python exception type corresponding to `HttpClientError::Tls`.
Reviewed By: quark-zju
Differential Revision: D26385846
fbshipit-source-id: c0df543032461de650a4d24c26c6b8aaab1abbb9
Summary:
Add a new `HttpClientError::Tls` variant specifically for TLS errors, separating them from other `curl::Error`s from libcurl.
To determine whether a particular `curl::Error` is a TLS error, we check both the error code and the contents of the error message.
Reviewed By: quark-zju
Differential Revision: D26385845
fbshipit-source-id: fd58f86a3a61fcfb845d19e262fdcb132dc7ec9f
