Commit Graph

15 Commits

Author SHA1 Message Date
Jan Mazur
3054c4eb63 send additional data identifying client
Summary: This will be used for rate limiting decisions. Also, could be logged to scuba tables to get more info about clients.

Reviewed By: quark-zju

Differential Revision: D28750197

fbshipit-source-id: 83f54e38f998c9dd824ef2d3834c777a44d0ffed
2021-06-07 06:38:37 -07:00
Arun Kulshreshtha
5ad2c03fb6 http-client: optionally convert certs to PKCS#12
Summary:
Convert client certificates (which are expected to be supplied as PEM files) into an in-memory PKCS#12 archive to pass into libcurl. This is necessary on certain platforms (such as Windows) whose native crypto APIs do not support loading PEM files.

This was previously landed as D27637069 (5b759a2b52), which unconditionally converted the certificates under the assumption that all major TLS backends support PKCS#12. That assumption is still true, but it did not account for the fact that libcurl itself is dynamically linked on some platforms (such as MacOS), and the system libcurl may be too old to support support in-memory certs (via `CURLOPT_SSLCERT_BLOB` added in libcurl version 7.71.0). This diff gates this feature behind the `http.convert-cert` config option, which we can selectively set on platforms where it is needed.

Reviewed By: mzr

Differential Revision: D28524444

fbshipit-source-id: 4af9cdd60b8ef3977ad81abdb8e406c63795e628
2021-05-19 10:39:45 -07:00
Arun Kulshreshtha
f738f65d28 http-client: add verbose option to requests
Summary: Add a new `http.verbose` config option that turns on verbose output for libcurl (similar to the output printed by `curl -v`). This can be very useful for debugging HTTP issues.

Reviewed By: DurhamG

Differential Revision: D27693304

fbshipit-source-id: 2ad7a08889f40ffbcd2f14ac9c21d70433629da4
2021-04-12 12:17:12 -07:00
Arun Kulshreshtha
3295f0f1fe hg-http: add support for --insecure flag
Summary:
Add a `set_global_config` function to `hg-http` that gets called prior to command execution. This function can be used to configure the global behavior of all HTTP requests by setting up a callback that modifies every `Request` prior to sending.

Right now, the use case for this is to support the `--insecure` flag, as can be seen in the code.

Reviewed By: quark-zju

Differential Revision: D27243638

fbshipit-source-id: 61a52c1f4b56fffd860c2a7e9c7b03e146b2240a
2021-03-31 12:53:46 -07:00
Arun Kulshreshtha
1e34440f74 http-client: allow on_new_request to configure Request
Summary:
Make the `RequestCreationEventListeners::new_request` event listener take an `&mut Request` instead of an `&mut RequestContext` as a parameter.

In the existing code (particularly in the `hg-http` crate), this event listener is used to configure the `RequestContext` for reporting progress. This diff just generalizes this idea, allowing the listener to modify the entire `Request`.

This is useful when we need to hijack request creation in `hg-http` to do Mercurial-specific configuration. The specific use case here is to disable TLS certificate checking when the global `--insecure` flag is set.

(Note that `http-client` itself is application-agnostic, so Mercurial specific configuration should not happen in this crate. This is why `hg-http` exists at all.)

Reviewed By: quark-zju

Differential Revision: D27242947

fbshipit-source-id: 019e19037642fe24acaa8c2917d446b91e7bcb26
2021-03-23 23:31:10 -07:00
Arun Kulshreshtha
c13c6cb4fd hg-http: tidy up imports and doc comment
Summary: Fix malformed module-level doc comment and rearrange import groups (based on the de facto Rust style conventions at FB).

Reviewed By: quark-zju

Differential Revision: D27243739

fbshipit-source-id: 21614d16e5fbfecd793939325ef52819db9a4cb8
2021-03-22 22:09:14 -07:00
Jun Wu
e9d5bed659 hg-http: add API to integrate with progress/model
Summary:
Add a way to register an IO time series showing total network I/O, and register
download progress bars for requests.

Reviewed By: andll

Differential Revision: D26853750

fbshipit-source-id: 42c4ab9724c60db9612a2bc419ca3bceaa667647
2021-03-11 17:19:14 -08:00
Jun Wu
0dbdbcb95a http-client: move with_stats_reporting to event listener
Summary:
This simplifies the code a bit and makes it look consistent with other event
listeners.

Reviewed By: kulshrax

Differential Revision: D26670318

fbshipit-source-id: f6eda9403bb6eb09a074544e672a45c84f38e2b1
2021-03-02 10:40:05 -08:00
Lukas Piatkowski
cd0b6d50e2 autocargo v1: changes to match autocargo v2 generation results.
Summary:
The changes (and fixes) needed were:
- Ignore rules that are not rust_library or thrift_library (previously only ignore rust_bindgen_library, so that binary and test dependencies were incorrectly added to Cargo.toml)
- Thrift package name to match escaping logic of `tools/build_defs/fbcode_macros/build_defs/lib/thrift/rust.bzl`
- Rearrange some attributes, like features, authors, edition etc.
- Authors to use " instead of '
- Features to be sorted
- Sort all dependencies as one instead of grouping third party and fbcode dependencies together
- Manually format certain entries from third-party/rust/Cargo.toml, since V2 formats third party dependency entries and V1 just takes them as is.

Reviewed By: zertosh

Differential Revision: D26544150

fbshipit-source-id: 19d98985bd6c3ac901ad40cff38ee1ced547e8eb
2021-02-19 11:03:55 -08:00
Lukas Piatkowski
87ddbe2f74 autocargo v1: update autocargo field format to allow transition to autocargo v2
Summary:
Autocargo V2 will use a more structured format for autocargo field
with the help of `cargo_toml` crate it will be easy to deserialize and handle
it.

Also the "include" field is apparently obsolete as it is used for cargo-publish (see https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields). From what I know this might be often wrong, especially if someone tries to publish a package from fbcode, then the private facebook folders might be shipped. Lets just not set it and in the new system one will be able to set it explicitly via autocargo parameter on a rule.

Reviewed By: ahornby

Differential Revision: D26339606

fbshipit-source-id: 510a01a4dd80b3efe58a14553b752009d516d651
2021-02-12 23:28:25 -08:00
Stefan Filip
f8461b8989 metrics: use dot . as metric separator
Summary:
Dot `.` is the common separator for the metrics aggregator that we use.
This adds some form of consistency.

Reviewed By: DurhamG

Differential Revision: D25968398

fbshipit-source-id: 194d2f33fe477fe5d768a9cd8f9f46f56445e3e8
2021-01-21 11:29:42 -08:00
Arun Kulshreshtha
f1aeffd67a hg-http: remove stats logging
Summary: Now that stats are logged to `tracing` by the `HttpClient` directly, we no longer need to log them here. This commit backs out D23858077 (613fbc858f) which added the logging.

Reviewed By: quark-zju

Differential Revision: D23919308

fbshipit-source-id: 23d3a12c5307bc4b84dd9ffd25bd376718e3cc91
2020-09-24 21:05:20 -07:00
Arun Kulshreshtha
613fbc858f hg-http: optionally print stats
Summary: Log HTTP stats to stderr to assist with ad-hoc debugging. Will not be printed unless `RUST_LOG` is set appropriately.

Reviewed By: quark-zju

Differential Revision: D23858077

fbshipit-source-id: 39acf3de3fd0ca4403a986eb5373a6a79f1d004a
2020-09-23 17:19:28 -07:00
Arun Kulshreshtha
31ceb7f0d1 hg-http: use autocargo
Summary: Onboard the crate onto autocargo.

Reviewed By: quark-zju

Differential Revision: D23858075

fbshipit-source-id: 7179ae0f9ca8a1d4e664d7eb5cb614940e2b2c30
2020-09-23 16:40:49 -07:00
Stefan Filip
c1ab6a4e92 http-client: add stats reporting hook
Summary:
Add `with_stats_reporting` to HttpClient. It takes a closure that will be
called with all `Stats` objects generated. We then use this function in
the hg-http crate to integrate with the metrics backend used in Mercurial.

Reviewed By: kulshrax

Differential Revision: D23577869

fbshipit-source-id: 5ac23f00183f3c3d956627a869393cd4b27610d4
2020-09-09 17:35:48 -07:00