Commit Graph

8 Commits

Author SHA1 Message Date
Jun Wu
443a8f33b3 indexedlog: move binary indexedlog_dump out
Summary:
It makes testing duplicated - now `cargo test` would try running tests on 2 entry points:
lib.rs and indexedlog_dump.rs.  Move it to a separate crate to solve the issue.

Reviewed By: markbt

Differential Revision: D13498266

fbshipit-source-id: 8abf07c1272dfa825ec7701fd8ea9e0d1310ec5f
2018-12-18 08:17:21 -08:00
Jun Wu
421c7b3f45 indexedlog: add a tool to dump indexedlog content
Summary: The tool can dump indexedlog content. Useful for manually investigating issues.

Reviewed By: DurhamG

Differential Revision: D13051387

fbshipit-source-id: 8687a1aa9dfb54776e80f184208c49da2492c34d
2018-12-06 14:57:52 -08:00
Jun Wu
42c3ef6eb6 indexedlog: add benchmark for "log"
Summary:
Some simple benchmark for "log". The initial result running from my devserver
looks like:

  log insertion                  33.146 ms
  log insertion with index      106.449 ms
  log flush                       9.623 ms
  log iteration (memory)         10.644 ms
  log iteration (disk)           11.517 ms
    13.75s user 3.61s system 97% cpu 17.778 total
  log insertion                  27.906 ms
  log insertion with index      107.683 ms
  log flush                      19.204 ms
  log iteration (memory)         10.239 ms
  log iteration (disk)           11.118 ms
    12.89s user 3.55s system 97% cpu 16.924 total
  log insertion                  31.645 ms
  log insertion with index      109.403 ms
  log flush                       9.416 ms
  log iteration (memory)         10.226 ms
  log iteration (disk)           10.757 ms
    13.07s user 3.02s system 97% cpu 16.423 total
  log insertion                  31.848 ms
  log insertion with index      109.332 ms
  log flush                      18.345 ms
  log iteration (memory)         10.709 ms
  log iteration (disk)           11.346 ms
    13.12s user 3.70s system 97% cpu 17.276 total
  log insertion                  29.665 ms
  log insertion with index      106.041 ms
  log flush                      16.159 ms
  log iteration (memory)         10.367 ms
  log iteration (disk)           11.110 ms
    12.99s user 3.27s system 97% cpu 16.717 total

Reviewed By: markbt

Differential Revision: D13015464

fbshipit-source-id: 035fee6c8b6d0bea4cfe194eed3d58ba4b5ebcb8
2018-12-06 14:57:52 -08:00
Jun Wu
616306543b codemod: use explicit versions in Cargo.toml
Summary:
This is done by running `fix-code.py`. Note that those strings are
semvers so they do not pin down the exact version. An API-compatiable upgrade
is still possible.

Reviewed By: ikostia

Differential Revision: D10213073

fbshipit-source-id: 82f90766fb7e02cdeb6615ae3cb7212d928ed48d
2018-11-15 18:54:06 -08:00
Jun Wu
7b9867ac12 crates: pin rand to 0.4 version
Summary:
`rand` 0.5 has too many breaking changes that the code is not ready to
migrate yet. So let's ping rand to 0.4. Ideally all dependencies in
Cargo.toml should avoid using "*". But for now `rand` is the only
troublemaker.

Note `rand 0.4` is a dependency of `quickcheck 0.6.2` so it's available.

Reviewed By: phillco, singhsrb

Differential Revision: D8158406

fbshipit-source-id: 417ae6807a2efc650acb8d82370964fab6531fdb
2018-05-25 09:51:19 -07:00
Jun Wu
8d7d4de8ee indexedlog: separate benchmarks
Summary:
The minibench framework does not provide benchmark filtering. So let's
separate benchmarks using different entry points.

Reviewed By: DurhamG

Differential Revision: D7440250

fbshipit-source-id: 11e7790a5074ebf4c08e33c312a490a66a921926
2018-04-13 21:51:42 -07:00
Jun Wu
9b9dd289e4 indexedlog: use minibench to do benchmark
Summary:
See the previous minibench diff for the motivation.

"failure" was removed from build dependencies since it's not used yet.

Run benchmark a few times. It seems the first several items are less stable
due to possibly warming up issues. Otherwise the result looks good enough.
The test also compiles and runs much faster.

```
base16 iterating 1M bytes       0.921 ms
index insertion                 4.804 ms
index flush                     5.104 ms
index lookup (memory)           2.929 ms
index lookup (disk)             1.767 ms
index clone (memory)            2.036 ms
index clone (disk)              0.010 ms

base16 iterating 1M bytes       0.853 ms
index insertion                 4.512 ms
index flush                     4.717 ms
index lookup (memory)           2.907 ms
index lookup (disk)             1.755 ms
index clone (memory)            1.856 ms
index clone (disk)              0.010 ms

base16 iterating 1M bytes       1.525 ms
index insertion                 4.577 ms
index flush                     4.901 ms
index lookup (memory)           2.800 ms
index lookup (disk)             1.790 ms
index clone (memory)            1.794 ms
index clone (disk)              0.010 ms

base16 iterating 1M bytes       0.768 ms
index insertion                 4.486 ms
index flush                     4.918 ms
index lookup (memory)           2.658 ms
index lookup (disk)             1.721 ms
index clone (memory)            1.763 ms
index clone (disk)              0.010 ms

base16 iterating 1M bytes       0.732 ms
index insertion                 4.489 ms
index flush                     4.792 ms
index lookup (memory)           2.689 ms
index lookup (disk)             1.739 ms
index clone (memory)            1.850 ms
index clone (disk)              0.009 ms

base16 iterating 1M bytes       1.124 ms
index insertion                 7.188 ms
index flush                     4.888 ms
index lookup (memory)           2.829 ms
index lookup (disk)             1.609 ms
index clone (memory)            2.642 ms
index clone (disk)              0.010 ms

base16 iterating 1M bytes       1.055 ms
index insertion                 4.683 ms
index flush                     4.996 ms
index lookup (memory)           2.782 ms
index lookup (disk)             1.710 ms
index clone (memory)            1.802 ms
index clone (disk)              0.009 ms
```

Reviewed By: DurhamG

Differential Revision: D7440249

fbshipit-source-id: 0f946ab184455acd40c5a38cf46ff94d9e3755c8
2018-04-13 21:51:42 -07:00
Jun Wu
0518016553 indexedlog: initial boilerplate
Summary:
First step of a storage-related building block that is in Rust. The goal is
to use it to replace revlog, obsstore and packfiles.

Extern crates that are likely useful are added to reduce future churns.

Reviewed By: DurhamG

Differential Revision: D7108434

fbshipit-source-id: 97ebd9ba69547d876dcecc05e604acdf9088877e
2018-04-13 21:51:24 -07:00