mirror of
https://github.com/facebook/sapling.git
synced 2025-01-06 04:43:19 +03:00
22e9000fc9
Summary:
Unfortunately required symbols are not exposed by lz4-sys. So we just declare
them ourselves.
Make sure it compresses better:
In [1]: c=open('/bin/bash').read();
In [2]: from mercurial.rust import lz4
In [3]: len(lz4.compress(c))
Out[3]: 762906
In [4]: len(lz4.compresshc(c))
Out[4]: 626970
While it's much slower for larger data (and compresshc is slower than pylz4):
Benchmarking (easy to compress data, 20MB)...
pylz4.compress: 10328.03 MB/s
rustlz4.compress_py: 9373.84 MB/s
pylz4.compressHC: 1666.80 MB/s
rustlz4.compresshc_py: 8298.57 MB/s
pylz4.decompress: 3953.03 MB/s
rustlz4.decompress_py: 3935.57 MB/s
Benchmarking (hard to compress data, 0.2MB)...
pylz4.compress: 4357.88 MB/s
rustlz4.compress_py: 4193.34 MB/s
pylz4.compressHC: 3740.40 MB/s
rustlz4.compresshc_py: 2730.71 MB/s
pylz4.decompress: 5600.94 MB/s
rustlz4.decompress_py: 5362.96 MB/s
Benchmarking (hard to compress data, 20MB)...
pylz4.compress: 5156.72 MB/s
rustlz4.compress_py: 5447.00 MB/s
pylz4.compressHC: 33.70 MB/s
rustlz4.compresshc_py: 22.25 MB/s
pylz4.decompress: 2375.42 MB/s
rustlz4.decompress_py: 5755.46 MB/s
Note python-lz4 was using an ancient version of lz4. So there could be differences.
Reviewed By: DurhamG
Differential Revision: D13528200
fbshipit-source-id: 6be1c1dd71f57d40dcffcc8d212d40a853583254
21 lines
351 B
TOML
21 lines
351 B
TOML
[package]
|
|
name = "lz4-pyframe"
|
|
version = "0.1.0"
|
|
authors = ["Facebook Source Control Team <sourcecontrol-dev@fb.com>"]
|
|
|
|
[dependencies]
|
|
byteorder = "1"
|
|
failure = "0"
|
|
libc = "0.2"
|
|
lz4-sys = "1"
|
|
|
|
[dev-dependencies]
|
|
minibench = { path = "../minibench" }
|
|
quickcheck = "0"
|
|
rand_chacha = "0.1.0"
|
|
rand_core = "0.3.0"
|
|
|
|
[[bench]]
|
|
name = "bench"
|
|
harness = false
|