2020-02-19 16:13:31 +03:00
|
|
|
[package]
|
|
|
|
name = "blobstore"
|
|
|
|
edition = "2018"
|
|
|
|
version = "0.1.0"
|
|
|
|
authors = ['Facebook']
|
|
|
|
license = "GPLv2+"
|
2020-03-06 12:28:21 +03:00
|
|
|
include = ["src/**/*.rs", "test/**/*.rs"]
|
|
|
|
|
|
|
|
[lib]
|
|
|
|
path = "src/lib.rs"
|
|
|
|
|
|
|
|
[[test]]
|
|
|
|
name = "blobstore_test"
|
|
|
|
path = "test/main.rs"
|
2020-02-19 16:13:31 +03:00
|
|
|
|
|
|
|
[dependencies]
|
|
|
|
context = { path = "../server/context" }
|
|
|
|
stats = { git = "https://github.com/facebookexperimental/rust-shed.git", branch = "master" }
|
2020-05-06 10:53:33 +03:00
|
|
|
abomonation = "0.7"
|
|
|
|
abomonation_derive = "0.5"
|
2020-02-19 16:13:31 +03:00
|
|
|
anyhow = "1.0"
|
2020-08-05 04:02:43 +03:00
|
|
|
auto_impl = "0.4"
|
mononoke: update internals to Bytes 0.5
Summary:
The Bytes 0.5 update left us in a somewhat undesirable position where every
access to our blobstore incurs an extra copy whenever we fetch data out of our
cache (by turning it from Bytes 0.5 into Bytes 0.4) — we also have quite a few
place where we convert in one direction then immediately into the other.
Internally, we can start using Bytes 0.5 now. For example, this is useful when
pulling data out of our blobstore and deserializing as Thrift (or conversely,
when serializing and putting it into our blobstore).
However, when we interface with Tokio (i.e. decoders & encoders), we still have
to use Bytes 0.4. So, when needed, we convert our Bytes 0.5 to 0.4 there.
The tradeoff idea is that we deal with more bytes internally than we end up
sending to clients, so doing the Bytes conversion closer to the point of
sending data to clients means less copies.
We can also start removing those once we migrate to Tokio 0.2 (and newer
versions of Hyper for HTTP services).
Changes that were required:
- You can't extend new bytes (because that implicitly copies). You need to use
BytesMut instead, which I did where that was necessary (I also added calls in
the Filestore to do that efficiently).
- You can't create bytes from a `&'a [u8]`, unless `'a` is `'static`. You need
to use `copy_from_slice` instead.
- `slice_to` and `slice_from` have been replaced by a `slice()` function that
takes ranges.
Reviewed By: StanislavGlebik
Differential Revision: D20121350
fbshipit-source-id: eb31af2051fd8c9d31c69b502e2f6f1ce2190cb1
2020-02-27 19:04:39 +03:00
|
|
|
bytes = { version = "0.5", features = ["serde"] }
|
2020-07-07 06:47:39 +03:00
|
|
|
futures = { version = "0.3.5", features = ["async-await", "compat"] }
|
2020-06-16 14:09:54 +03:00
|
|
|
futures-old = { package = "futures", version = "0.1" }
|
2020-08-04 16:49:52 +03:00
|
|
|
serde = { version = "1.0", features = ["derive", "rc"] }
|
|
|
|
serde_derive = "1.0"
|
2020-10-06 23:03:46 +03:00
|
|
|
strum = "0.17"
|
|
|
|
strum_macros = "0.17"
|
2020-02-19 16:13:31 +03:00
|
|
|
thiserror = "1.0"
|
third-party/rust: import async-compression + update zstd
Summary:
This imports the async-compression crate. We have an equivalent-ish in
common/rust, but it targets Tokio 0.1, whereas this community-supported crate
targets Tokio 0.2 (it offers a richer API, notably in the sense that we
can use it for Streams, whereas the async-compression crate we have is only for
AsyncWrite).
In the immediate term, I'd like to use this for transfer compression in
Mononoke's LFS Server. In the future, we might also use it in Mononoke where we
currently use our own async compression crate when all that stuff moves to
Tokio 0.2.
Finally, this also updates zstd: the version we link to from tp2 is actually
zstd 1.4.5, so it's a good idea to just get the same version of the zstd crate.
The zstd crate doesn't keep a great changelog, so it's hard to tell what has changed.
At a glance, it looks like the answer is not much, but I'm going to look to Sandcastle
to root out potential issues here.
Reviewed By: StanislavGlebik
Differential Revision: D23652335
fbshipit-source-id: e250cef7a52d640bbbcccd72448fd2d4f548a48a
2020-09-15 17:57:18 +03:00
|
|
|
zstd = "=0.5.3+zstd.1.4.5"
|
2020-02-19 16:13:31 +03:00
|
|
|
|
|
|
|
[dev-dependencies]
|
2020-06-20 11:04:21 +03:00
|
|
|
cacheblob = { path = "cacheblob" }
|
2020-03-06 12:28:21 +03:00
|
|
|
fileblob = { path = "fileblob" }
|
|
|
|
memblob = { path = "memblob" }
|
|
|
|
mononoke_types = { path = "../mononoke_types" }
|
2020-06-18 03:52:59 +03:00
|
|
|
fbinit = { git = "https://github.com/facebookexperimental/rust-shed.git", branch = "master" }
|
2020-10-07 22:08:51 +03:00
|
|
|
itertools = "0.8"
|
2020-03-06 12:28:21 +03:00
|
|
|
tempdir = "0.3"
|
2020-02-19 16:13:31 +03:00
|
|
|
tokio-compat = "0.1"
|