Commit Graph

24 Commits

Author SHA1 Message Date
Facebook Community Bot
9728bf52e9 Re-sync with internal repository 2022-02-07 12:59:09 +00:00
Facebook Community Bot
88edb37132 Re-sync with internal repository 2022-02-03 21:43:02 +00:00
Jun Wu
2a99866d7b codemod: update license headers
Summary:
The "Portions" license cannot be updated automatically. So this is a manual
update using:

  sd -s 'Portions Copyright (c) Facebook, Inc. and its affiliates.' 'Portions Copyright (c) Meta Platforms, Inc. and affiliates.' `rg -l Facebook`
  sd -s 'Copyright (c) Facebook, Inc. and its affiliates.' 'Copyright (c) Meta Platforms, Inc. and affiliates.' `rg -l Facebook`

Differential Revision: D33420114

fbshipit-source-id: 49ae00a7b62e3b8cc6c5dd839b3c104a75e72a56
2022-01-05 14:43:32 -08:00
David Tolnay
2ea696cb5a Regenerate Cargo.tomls to 2021 edition
Summary: Generated by `sed -i 's/edition = "2018"/edition = "2021"/' common/rust/cargo_from_buck/project_configs/*.toml` and `cargo autocargo` following the autocargo code change in D31830985 and its corresponding MSDK rebuild in D31832311.

Reviewed By: zertosh

Differential Revision: D31832698

fbshipit-source-id: db79024cad311467d5b7c13b7abedc6464f09366
2021-10-21 12:40:28 -07:00
Jun Wu
ae33ddb533 rustfmt: format codebase
Summary:
See previous diff for context. The reorders the `use` statements to make
things a bit more consistent.

Reviewed By: yancouto

Differential Revision: D31751166

fbshipit-source-id: 09c4ddba69ddeae1fc1b996d93fbfa6063e3407d
2021-10-19 11:07:24 -07:00
Jun Wu
d2fd54d036 rustfmt: format part of codebase
Summary:
The current rustfmt we use cannot handle `#[cfg(...)] use ...::{...}` properly.
Let's format the files that do not have `#[cfg] use` or are generated first.

The change is done by Ruby code:

  list = Dir['{lib,edenscmnative}/**/*.rs'].select {|p|!File.read(p)["generated"]}
  system 'rustfmt', *list

  $fn = nil
  reverts = `hg diff`.lines.map do |l|
    $fn = l[/diff --git a\/fbcode\/eden\/scm\/([^ ]*) b\//, 1] || $fn
    $fn if l[/-\s*#\[cfg/]
  end.compact.uniq
  system 'hg', 'revert', *reverts

Reviewed By: yancouto

Differential Revision: D31751037

fbshipit-source-id: dbe0b39e35345b7d1c54cbdf67b8aafd19df9c70
2021-10-19 11:07:24 -07:00
Davide Cavalca
5ad11146fc Bump some dependencies for below
Summary: Bump in preparation for a new release

Reviewed By: brianc118

Differential Revision: D31657479

fbshipit-source-id: eeb2a08c54f6db75b6e515ca0966637f31b906a8
2021-10-14 17:36:29 -07:00
Jun Wu
9f1499437b minibytes: deserialize from more bytes types
Summary:
Accept owned bytes (`Vec<u8>`). This will make `Serde<Bytes>` work in Python
bindings.

Reviewed By: yancouto

Differential Revision: D31615717

fbshipit-source-id: ad200d359b9282fa84d1698afdde8241fc288905
2021-10-13 13:38:08 -07:00
Victor Gao
ae87b82eaf update rand and quickcheck
Summary:
This updates the following crates to their latest versions:
- `rand`: 0.7 => 0.8
- `quickcheck`: 0.9 => 1.0

Both crates introduced some breaking changes, so affected clients had to be fixed accordingly. Most changes are rather mechanical and shouldn't change the existing logic. In addition, a few buggy property tests were uncovered, presumably due to `quicktest` becoming smarter with its choice of inputs in the newer version, and the fixes are included in this diff.

Reviewed By: yancouto

Differential Revision: D31115820

fbshipit-source-id: 60a61dfac3236fd93cd4f03b86506654d81d330f
2021-09-29 13:59:49 -07:00
Davide Cavalca
b82c5672fc Update several rust crate versions
Summary: Update versions for several of the crates we depend on.

Reviewed By: danobi

Differential Revision: D29165283

fbshipit-source-id: baaa9fa106b7dad000f93d2eefa95867ac46e5a1
2021-06-17 16:38:19 -07:00
Thomas Orozco
846a983d67 thrift/lib/rust: update to Bytes 1.x
Summary:
Like it says in the title. The API between Bytes 1.x has changed a little bit,
but the concepts are basically the same, so we just need to change the
callsites that were calling `bytes()` and have them ask for `chunk()` instead.

This diff attempts to be as small as it can (and it's already quite big). I
didn't attempt to update *everything*: I only updated whatever was needed to
keep `common/rust/tools/scripts/check_all.sh` passing.

However, there are a few changes that fall out of this. I'll outline them here:

## `BufExt`

One little caveat is the `copy_to_bytes` we had on `BufExt`. This was
introduced into Bytes 1.x (under that name), but we can't use it here directly.

The reason we can't is because the instance we have is a `Cursor<Bytes>`, which
receives an implementation of `copy_from_bytes` via:

```
impl<T: AsRef<[u8]>> Buf for std::io::Cursor<T>
```

This means that implementation isn't capable of using the optimized
`Bytes::copy_from_bytes` which doesn't do a copy at all. So, instead, we need
to use a dedicated method on `Cursor<Bytes>`: `copy_or_reuse_bytes`.

## Calls to `Buf::to_bytes()`

This method is gone in Bytes 1.x, and replaced by the idiom
`x.copy_to_bytes(x.remaining())`, so I updated callsites of `to_bytes()`
accordingly.

## `fbthrift_ext`

This set of crates provides transports for Thrift calls that rely on Tokio 0.2
for I/O. Unfortunately, Tokio 0.2 uses Bytes 0.5, so that doesn't work well.

For now, I included a copy here (there was only one required, when reading from
the socket). This can be removed if we update the whole `fbthrift_ext` stack to
Bytes 1.x. fanzeyi had been wanting to update this to Tokio 1.x, but was blocked on `thrift/lib/rust` using Bytes 0.5, and confirmed that the overhead of a copy here is fine (besides, this code can now be updated to Tokio 1.x to remove the copy).

## Crates using both Bytes 0.5 & Bytes 1.x

This was mostly the case in Mononoke. That's no coincidence: this is why I'm
working on this. There, I had to make changes that consist of removing Bytes
0.5 to Bytes 1.x copies.

## Misuse of `Buf::bytes()`

Some places use `bytes()` when they probably mean to use `copy_to_bytes()`. For
now, I updated those to use `chunk()`, which keeps the behavior the same but
keeps the code buggy. I filed T91156115 to track fixing those (in all
likelihood I will file tasks for the relevant teams).

Reviewed By: dtolnay

Differential Revision: D28537964

fbshipit-source-id: ca42a614036bc3cb08b21a572166c4add72520ad
2021-05-20 09:44:41 -07:00
Meyer Jacobs
d0464a9838 minibytes: Debug format Bytes as a bytestring literal with ascii escapes
Summary:
Modify the `Debug` implementation for `minibytes::Bytes` to use `std::ascii::escape_default` to debug print a `Bytes` as an ascii-escaped bytestring.

For comparison, the `bytes` crate `Bytes` type provides the same functionality, though it doesn't use the standard library `escape_default` function: https://docs.rs/bytes/1.0.1/src/bytes/fmt/debug.rs.html#39-43

This change greatly improves the output of the `debugscmstore` command. If we don't want to make this the default behavior, we can provide a formatting wrapper type or I can specialize the output in `debugscmstore`, but I can't see any real downsides, especially given the `bytes` crate does the same thing, and we have a similar specialization for `HgId` (hex format in that case).

Reviewed By: quark-zju

Differential Revision: D27642721

fbshipit-source-id: 8faba421fa5082a2098b13ef7d286e05eccb6400
2021-04-09 14:43:13 -07:00
Jun Wu
16564f94c9 minibytes: into_vec cannot use fast path if the vec is sliced
Summary:
Add a check to ensure the `into_vec()` fast path do not take the full vec if
the `Bytes` is sliced.

Reviewed By: andll

Differential Revision: D26966453

fbshipit-source-id: 538dfb8ca2f01a46d1ede7b98b7f0a30fc7a786e
2021-03-10 17:16:53 -08:00
Lukas Piatkowski
ad106958f2 eden/scm/lib: autogenerate all Cargo.toml files with autocargo
Summary: This diff removes the split between manually managed and autocargo managed Cargo.toml files in `eden/scm/lib`, now all files are autogenerated.

Reviewed By: quark-zju

Differential Revision: D26830884

fbshipit-source-id: 3a5d8409a61347c7650cc7d8192fa426c03733dc
2021-03-05 04:29:49 -08:00
Jun Wu
e817eb52e9 minibytes: add Bytes::into_vec
Summary:
Make it possible to convert Bytes to `Vec<u8>` in a zero-copy way if possible.

This will make it possible to convert `minibytes::Bytes` to `bytes::Bytes` in a
zero-copy way if possible. Practically, it might be useful for some
revisionstore -> edenapi/types usecases.

Reviewed By: kulshrax

Differential Revision: D26237922

fbshipit-source-id: 28d620f303511099df77f79256d98abb1010f665
2021-02-05 13:57:49 -08:00
Jun Wu
16a92c5cc1 minibytes: add downcast_mut
Summary: This API will be used in the next diff.

Reviewed By: DurhamG

Differential Revision: D26237923

fbshipit-source-id: 69438072c2edef1ce28ceef3b8b723f015f54ff5
2021-02-05 13:57:48 -08:00
Jun Wu
06accd8674 minibytes: implement From<bytes::Bytes>
Summary:
Make it possible to construct minibytes::Bytes via bytes::Bytes.
This will be used later.

Reviewed By: kulshrax

Differential Revision: D26232990

fbshipit-source-id: 36af6f28fd08eb457de8b9223235ec038ac3ef14
2021-02-03 20:22:32 -08:00
Xavier Deguillard
deffd9a477 minibytes: address clippy warnings
Summary: Using ptr.add is shorter and preferred to ptr.offset.

Reviewed By: quark-zju

Differential Revision: D20452752

fbshipit-source-id: 1dc2fdbc392267d2d690673c10dcc161ecd00dfa
2020-03-16 14:58:22 -07:00
Jun Wu
c4ec99ded4 minibytes: implement Text
Summary:
Text is a reference-counted shared String.
It's similar to Bytes but works for utf-8 strings.

The motivation is to replace configparser's use of Bytes to Text.

Reviewed By: markbt

Differential Revision: D20432940

fbshipit-source-id: ef990255d269e60d433c6520819f60ccdcbe488f
2020-03-13 10:51:41 -07:00
Jun Wu
7895e70dcf minibytes: make Bytes abstract
Summary: This makes it possible to implement "Text". See the next diff.

Reviewed By: markbt

Differential Revision: D20432943

fbshipit-source-id: 94b3810ab205c260d33f57bd637e4accc3ee871d
2020-03-13 10:51:40 -07:00
Jun Wu
e9b14b3608 minibytes: implement From<&'static {str,[u8]}>
Summary:
This makes the API easier to use.

Practically this makes it easier for configparser to migrate to minibytes.

Reviewed By: markbt

Differential Revision: D20432942

fbshipit-source-id: ad08eb118d2216054dc24c86b0b129ae82b9d17c
2020-03-13 10:51:40 -07:00
Jun Wu
abb8ccb346 minibytes: add serde support
Summary:
See also https://github.com/serde-rs/bytes/.

This will be used in the `dag` crate.

Reviewed By: DurhamG

Differential Revision: D19770858

fbshipit-source-id: 2a870a564e0ceecdc7a4667853b2b2a5ea4ce6e3
2020-02-07 14:21:39 -08:00
Jun Wu
8029cd3878 minibytes: port benchmark from tokio/bytes
Summary:
Performance looks okay comparing with tokio/bytes v0.5.4:

minibytes:

  test clone_arc_vec        ... bench:      16,542 ns/iter (+/- 1,524)
  test clone_shared         ... bench:      16,211 ns/iter (+/- 596)
  test clone_static         ... bench:       1,437 ns/iter (+/- 502)
  test deref_shared         ... bench:         367 ns/iter (+/- 101)
  test deref_static         ... bench:         366 ns/iter (+/- 1)
  test deref_unique         ... bench:         367 ns/iter (+/- 4)
  test from_long_slicd      ... bench:          91 ns/iter (+/- 18) = 1406 MB/s
  test slice_empty          ... bench:      10,382 ns/iter (+/- 104)
  test slice_short_from_arc ... bench:      23,823 ns/iter (+/- 1,411)

tokio/bytes:

  test clone_arc_vec        ... bench:      16,213 ns/iter (+/- 1,864)
  test clone_shared         ... bench:      18,685 ns/iter (+/- 634)
  test clone_static         ... bench:       3,983 ns/iter (+/- 163)
  test deref_shared         ... bench:         366 ns/iter (+/- 26)
  test deref_static         ... bench:         373 ns/iter (+/- 36)
  test deref_unique         ... bench:         391 ns/iter (+/- 33)
  test from_long_slice      ... bench:          67 ns/iter (+/- 7) = 1910 MB/s
  test slice_empty          ... bench:      15,149 ns/iter (+/- 1,708)
  test slice_short_from_arc ... bench:      36,541 ns/iter (+/- 3,485)

clone_static is faster because minibytes don't call into vtable's clone.
from_long_slice is slower because minibytes uses Arc unconditionally while bytes
can avoid Arc overhead if refcount is 1.

Reviewed By: DurhamG

Differential Revision: D19770857

fbshipit-source-id: 5bafcc57a38c68baccfcafd3906f1a47b2bf4530
2020-02-07 14:21:39 -08:00
Jun Wu
108f1c947a minibytes: minimalist zero-copy Bytes with mmap support
Summary:
This crate provides the core features of the commonly known `Bytes` crate:
zero-copy slicing and cloning, while also supports mmap-backed buffers.

The main motivation is to replace `Mmap` in `indexedlog`. That has multiple
benefits:
- Handles 0-sized mmap more cleanly.
- Handles clones more cleanly.
- Gain the flexibility to zero-copy data without lifetime / reference.
- Gain the flexibility to switch to non-mmap data.

The `bytes::Bytes` crate does not yet support mmap buffers as of its latest
release (0.5.4).

Implementation wise, `minibytes::Bytes` uses `Option<Arc<dyn Trait>>` for the
"trait object". This makes implementing the mmap storage just one line.
`bytes 0.5.4` re-invents the "trait object" manually using unsafe code. It requires
about 50 lines to implement the mmap storage (in D19756122).

Reviewed By: xavierd

Differential Revision: D19770856

fbshipit-source-id: 8cfa7052a18ac2e0cd6348b77d5e2a4acc61195c
2020-02-07 14:21:38 -08:00