Jun Wu
55b6331aa4
indexedlog: add more benchmarks
...
Summary:
Add benchmarks about index sizes, and a benchmark of insertion using key
references.
An example `cargo bench` result running on my devserver looks like:
index insertion (owned key) 3.551 ms
index insertion (referred key) 3.713 ms
index flush 20.648 ms
index lookup (memory) 1.087 ms
index lookup (disk, no verify) 2.041 ms
index lookup (disk, verified) 4.347 ms
index size (owned key) 886010
index size (referred key) 534298
Reviewed By: markbt
Differential Revision: D13027879
fbshipit-source-id: 70644c504026ffee2122d857d5035f5b7eea4f42
2018-12-06 14:57:52 -08:00
Jun Wu
f9fb60337a
minibench: add a simple library to do benchmark
...
Summary:
It's sad to find that existing Rust benchmark frameworks do not fit well in
our simple benchmark purpose. The benchmark library shipped with Rust [1] has
been in "nightly-only" for long. Third-party choices like "criterion.rs" does
too many things and misses certain small features. Namely, indexedlog wants:
- More stable benchmark result. This means not picking the average time,
but the "best" time among all runs, like what Mercurial does.
- Do not measure setup cost from repetitive runs. As in D7404532, do not
clone the index, and do not have separate "clone" benchmarks.
- Faster benchmarks. This means getting rid of unused parts like calling
gnuplot.
Besides, having the test framework to be lightweight also helps compilation
time. Looking at `indexedlog`'s dependencies (with unused "failure"
removed), 70% of them are from `criterion.rs`.
```
indexedlog v0.1.0 (lib/indexedlog)
[dependencies]
|-- atomicwrites v0.1.5
| [dependencies]
| |-- nix v0.9.0
| | [dependencies]
| | |-- bitflags v0.9.1
| | |-- cfg-if v0.1.2
| | |-- libc v0.2.39
| | `-- void v1.0.2
| `-- tempdir v0.3.6
| [dependencies]
| |-- rand v0.4.2
| | [dependencies]
| | `-- libc v0.2.39 (*)
| `-- remove_dir_all v0.3.0
| [dependencies]
| |-- kernel32-sys v0.2.2
| | [dependencies]
| | `-- winapi v0.2.8
| | [build-dependencies]
| | `-- winapi-build v0.1.1
| `-- winapi v0.2.8 (*)
|-- byteorder v1.2.1
|-- fs2 v0.4.3
| [dependencies]
| `-- libc v0.2.39 (*)
|-- memmap v0.6.2
| [dependencies]
| `-- libc v0.2.39 (*)
|-- twox-hash v1.1.0
| [dependencies]
| `-- rand v0.3.22
| [dependencies]
| |-- libc v0.2.39 (*)
| `-- rand v0.4.2 (*)
`-- vlqencoding v0.1.0 (lib/vlqencoding)
[dev-dependencies]
|-- criterion v0.2.1
| [dependencies]
| |-- atty v0.2.8
| | [dependencies]
| | `-- libc v0.2.39 (*)
| |-- clap v2.31.1
| | [dependencies]
| | |-- ansi_term v0.11.0
| | |-- atty v0.2.8 (*)
| | |-- bitflags v1.0.1
| | |-- strsim v0.7.0
| | |-- textwrap v0.9.0
| | | [dependencies]
| | | `-- unicode-width v0.1.4
| | |-- unicode-width v0.1.4 (*)
| | `-- vec_map v0.8.0
| |-- criterion-plot v0.2.1
| | [dependencies]
| | |-- byteorder v1.2.1 (*)
| | |-- cast v0.2.2
| | `-- itertools v0.7.7
| | [dependencies]
| | `-- either v1.4.0
| |-- criterion-stats v0.2.1
| | [dependencies]
| | |-- cast v0.2.2 (*)
| | |-- num-traits v0.2.1
| | |-- num_cpus v1.8.0
| | | [dependencies]
| | | `-- libc v0.2.39 (*)
| | |-- rand v0.4.2 (*)
| | `-- thread-scoped v1.0.2
| |-- failure v0.1.1
| | [dependencies]
| | |-- backtrace v0.3.5
| | | [dependencies]
| | | |-- backtrace-sys v0.1.16
| | | | [dependencies]
| | | | `-- libc v0.2.39 (*)
| | | | [build-dependencies]
| | | | `-- cc v1.0.8
| | | |-- cfg-if v0.1.2 (*)
| | | |-- libc v0.2.39 (*)
| | | `-- rustc-demangle v0.1.7
| | `-- failure_derive v0.1.1
| | [dependencies]
| | |-- quote v0.3.15
| | |-- syn v0.11.11
| | | [dependencies]
| | | |-- quote v0.3.15 (*)
| | | |-- synom v0.11.3
| | | | [dependencies]
| | | | `-- unicode-xid v0.0.4
| | | `-- unicode-xid v0.0.4 (*)
| | `-- synstructure v0.6.1
| | [dependencies]
| | |-- quote v0.3.15 (*)
| | `-- syn v0.11.11 (*)
| |-- failure_derive v0.1.1 (*)
| |-- handlebars v0.31.0
| | [dependencies]
| | |-- lazy_static v1.0.0
| | |-- log v0.4.1
| | | [dependencies]
| | | `-- cfg-if v0.1.2 (*)
| | |-- pest v1.0.6
| | |-- pest_derive v1.0.6
| | | [dependencies]
| | | |-- pest v1.0.6 (*)
| | | |-- quote v0.3.15 (*)
| | | `-- syn v0.11.11 (*)
| | |-- quick-error v1.2.1
| | |-- regex v0.2.10
| | | [dependencies]
| | | |-- aho-corasick v0.6.4
| | | | [dependencies]
| | | | `-- memchr v2.0.1
| | | | [dependencies]
| | | | `-- libc v0.2.39 (*)
| | | |-- memchr v2.0.1 (*)
| | | |-- regex-syntax v0.5.3
| | | | [dependencies]
| | | | `-- ucd-util v0.1.1
| | | |-- thread_local v0.3.5
| | | | [dependencies]
| | | | |-- lazy_static v1.0.0 (*)
| | | | `-- unreachable v1.0.0
| | | | [dependencies]
| | | | `-- void v1.0.2 (*)
| | | `-- utf8-ranges v1.0.0
| | |-- serde v1.0.33
| | `-- serde_json v1.0.11
| | [dependencies]
| | |-- dtoa v0.4.2
| | |-- itoa v0.3.4
| | |-- num-traits v0.2.1 (*)
| | `-- serde v1.0.33 (*)
| |-- itertools v0.7.7 (*)
| |-- itertools-num v0.1.1
| | [dependencies]
| | `-- num-traits v0.1.43
| | [dependencies]
| | `-- num-traits v0.2.1 (*)
| |-- log v0.4.1 (*)
| |-- serde v1.0.33 (*)
| |-- serde_derive v1.0.33
| | [dependencies]
| | |-- proc-macro2 v0.2.3
| | | [dependencies]
| | | `-- unicode-xid v0.1.0
| | |-- quote v0.4.2
| | | [dependencies]
| | | `-- proc-macro2 v0.2.3 (*)
| | |-- serde_derive_internals v0.21.0
| | | [dependencies]
| | | |-- proc-macro2 v0.2.3 (*)
| | | `-- syn v0.12.14
| | | [dependencies]
| | | |-- proc-macro2 v0.2.3 (*)
| | | |-- quote v0.4.2 (*)
| | | `-- unicode-xid v0.1.0 (*)
| | `-- syn v0.12.14 (*)
| |-- serde_json v1.0.11 (*)
| `-- simplelog v0.5.0
| [dependencies]
| |-- chrono v0.4.0
| | [dependencies]
| | |-- num v0.1.42
| | | [dependencies]
| | | |-- num-integer v0.1.36
| | | | [dependencies]
| | | | `-- num-traits v0.2.1 (*)
| | | |-- num-iter v0.1.35
| | | | [dependencies]
| | | | |-- num-integer v0.1.36 (*)
| | | | `-- num-traits v0.2.1 (*)
| | | `-- num-traits v0.2.1 (*)
| | `-- time v0.1.39
| | [dependencies]
| | `-- libc v0.2.39 (*)
| | [dev-dependencies]
| | `-- winapi v0.3.4
| |-- log v0.4.1 (*)
| `-- term v0.4.6
|-- quickcheck v0.6.2
| [dependencies]
| |-- env_logger v0.5.6
| | [dependencies]
| | |-- atty v0.2.8 (*)
| | |-- humantime v1.1.1
| | | [dependencies]
| | | `-- quick-error v1.2.1 (*)
| | |-- log v0.4.1 (*)
| | |-- regex v0.2.10 (*)
| | `-- termcolor v0.3.5
| |-- log v0.4.1 (*)
| `-- rand v0.4.2 (*)
|-- rand v0.4.2 (*)
`-- tempdir v0.3.6 (*)
```
[1]: https://github.com/rust-lang/rust/issues/29553
Reviewed By: DurhamG
Differential Revision: D7440254
fbshipit-source-id: 53cdbd470945388db96702ab771a3f73b456da37
2018-04-13 21:51:42 -07:00