Arun Kulshreshtha a1be9e6a7f http-client: avoid quadratic behavior when deseralizing large CBOR values ... Summary: EdenAPI makes heavy use of streaming HTTP responses consisting of a series of serialized CBOR values. In order to process the data in a streaming manner, we use the `CborStream` combinator, which attempts to deserialize the CBOR values as they are received. `CborStream` hits a pathological case when it receives a very large CBOR value. Previously, it would always buffer the input stream into 1 MB chunks, and attempt to deserialize whenever a new chunk was received. In the case of downloading values that are >1GB in size, this meant potentially thousands of wasted deserialization attempts. In practice, this meant that EdenAPI would hang when receiving the content of large files. To address this problem, this diff adds a simple heuristic: If a partial CBOR value exceeds the current buffer size, double the size threshold before attempting to deserialize again. This reduces the pathological case from `O(n^2)` to `O(log(n))` (with some caveats, described in the comment in the code). Reviewed By: krallin Differential Revision: D27759698 fbshipit-source-id: 67882c31ce95a934b96c61f1c72bd97cad942d2e		2021-04-14 17:34:01 -07:00
..
src	http-client: avoid quadratic behavior when deseralizing large CBOR values	2021-04-14 17:34:01 -07:00
Cargo.toml	update tokio to 1.4.0	2021-03-31 10:44:20 -07:00