facebook/sapling
1
1
Fork 0
mirror of https://github.com/facebook/sapling.git synced 2024-10-11 01:07:15 +03:00
Code Issues Projects Releases Wiki Activity
73765b649b
sapling/eden/scm/lib/indexedlog/Cargo.toml

37 lines
577 B
TOML
Raw Normal View History

indexedlog: initial boilerplate Summary: First step of a storage-related building block that is in Rust. The goal is to use it to replace revlog, obsstore and packfiles. Extern crates that are likely useful are added to reduce future churns. Reviewed By: DurhamG Differential Revision: D7108434 fbshipit-source-id: 97ebd9ba69547d876dcecc05e604acdf9088877e
2018-03-10 05:57:11 +03:00
[package]
name = "indexedlog"
version = "0.1.0"
indexedlog: migrate to Rust 2018 Summary: Used `cargo fix --edition`. Removed some `mut`s according to rustc warnings. Differential Revision: D14718308 fbshipit-source-id: 94e3c3f8e47143ede767fe883fdb5e9602b12854
2019-04-04 08:12:16 +03:00
edition = "2018"
indexedlog: initial boilerplate Summary: First step of a storage-related building block that is in Rust. The goal is to use it to replace revlog, obsstore and packfiles. Extern crates that are likely useful are added to reduce future churns. Reviewed By: DurhamG Differential Revision: D7108434 fbshipit-source-id: 97ebd9ba69547d876dcecc05e604acdf9088877e
2018-03-10 05:57:11 +03:00
[dependencies]
codemod: use explicit versions in Cargo.toml Summary: This is done by running `fix-code.py`. Note that those strings are semvers so they do not pin down the exact version. An API-compatiable upgrade is still possible. Reviewed By: ikostia Differential Revision: D10213073 fbshipit-source-id: 82f90766fb7e02cdeb6615ae3cb7212d928ed48d
2018-11-16 05:18:31 +03:00
byteorder = "1.2.7"
fs2 = "0.4.3"
indexedlog: try to use symlink for atomic_write on unix Summary: The change is in theory not necessary. However it improves the reliability on OS crashes a bit, and can potentially workaround some bugs in filesystems (as we saw in production where the atomic-written files are empty and the system didn't crash). The idea is, the `symlink` syscall does the file creation and "content" writing together, while there is no way to create a file and write specific content in one syscall. Note that the C symlink call uses 0-terminated string, and the Rust stdlib exports it as accepting `Path`. To be safe, we encode binary or non-utf8 content using `hex`. For downgrade safety, the write path does not use symlink by default unless format.use-symlink-atomic-write is set to true. This makes downgrade possible: the read path is rolled out first, then we can turn on and off the write path. The indexedlog Rust unit tests and test-doctor.t are migrated to use the new symlink code paths. Reviewed By: DurhamG Differential Revision: D20153864 fbshipit-source-id: c31bd4287a8d29575180fbcf7227d2b04c4c1252
2020-03-04 18:20:52 +03:00
hex = "0.4"
indexedlog: add a way to chown gid for created directory Summary: This adds some global state to specify default chown/chmod for indexedlog-created directories. Reviewed By: xavierd Differential Revision: D17870604 fbshipit-source-id: cf8d8383e0d4e4d33b452e44f15fddcc40f36484
2019-10-11 05:51:53 +03:00
libc = "0.2"
codemod: use explicit versions in Cargo.toml Summary: This is done by running `fix-code.py`. Note that those strings are semvers so they do not pin down the exact version. An API-compatiable upgrade is still possible. Reviewed By: ikostia Differential Revision: D10213073 fbshipit-source-id: 82f90766fb7e02cdeb6615ae3cb7212d928ed48d
2018-11-16 05:18:31 +03:00
memmap = "0.7.0"
indexedlog: switch from bytes to minibytes Summary: This makes it possible to use `Bytes` for mmap buffers. The changes are because `minibytes::Bytes` does not implement `From<&[u8]>` with the intention to make slice copy explicit. Reviewed By: xavierd Differential Revision: D19818719 fbshipit-source-id: c34ee451bfd2dc7bcbbcebd52a76444b6c236849
2020-02-13 00:55:35 +03:00
minibytes = { path = "../minibytes" }
indexedlog: make logs in RotateLog lazy Summary: Previously, RotateLog loads as many logs as it can during initialization or sync. However, that could be undesirable because loading too many logs can take time. Make log loading lazy except for the `latest` log to reduce overhead. Reviewed By: xavierd Differential Revision: D17740792 fbshipit-source-id: cde4c1799ed55d390dadaa5bd34f3d2c6d0e1cf7
2019-10-05 06:33:58 +03:00
once_cell = "1"
rust: Fix all fbcode warnings Summary: With this change, all Rust code in fbcode compiles with rustc 1.39 without warnings. - Change UB uses of std::mem::uninitialized to MaybeUninit - Remove unused `mut` - Remove stray semicolons - Remove unused imports - Handle or explicitly ignore ignored errors - Ignore dead code warnings in half-written projects - Suppress warnings in bindgen-generated code Reviewed By: andrewjcg Differential Revision: D18363394 fbshipit-source-id: 2f9d587887aa0f8039c63202533bffba9006374f
2019-11-07 20:28:46 +03:00
rand = "0.7"
indexedlog: replace atomicwrites with tempfile Summary: Some tests added by upcoming diffs were timing out while they don't seem that expensive. I tracked it down to the use of `fsync` in atomicwrites. In our case, we don't need `fsync`. `fsync` is useful for making sure the order of file writes is desired even in case of system crash. For example, making sure the "primary" log file is written before writing the "meta" file. That's too expensive (esp. on filesystems like ext4) for our usecase. Indexedlog is designed to make sure data corruption can be detected, and there can be a "reasonable" way to recover (ex. by deleting all indexes, scanning through entries and re-inserting them in a new log), not to fight against OS crashes. `cargo bench` change on a btrfs filesystem: Before: index flush 42.570 ms log flush 7.712 ms After: index flush 36.485 ms log flush 1.609 ms Differential Revision: D14759304 fbshipit-source-id: 66b95d10040cf1480367b767811dfabee5e27ffe
2019-04-04 08:12:16 +03:00
tempfile = "3.0.7"
indexedlog: add some tracing spans Summary: Add some spans around `open` and `sync` so (maybe interesting) indexedlog operations are logged. Reviewed By: sfilipco Differential Revision: D18018378 fbshipit-source-id: e46affe675e6b0435b3aa7d5160a3211d2eb72e1
2019-11-05 02:49:48 +03:00
tracing = "0.1"
indexedlog: make logs in RotateLog lazy Summary: Previously, RotateLog loads as many logs as it can during initialization or sync. However, that could be undesirable because loading too many logs can take time. Make log loading lazy except for the `latest` log to reduce overhead. Reviewed By: xavierd Differential Revision: D17740792 fbshipit-source-id: cde4c1799ed55d390dadaa5bd34f3d2c6d0e1cf7
2019-10-05 06:33:58 +03:00
twox-hash = "1"
indexedlog: initial boilerplate Summary: First step of a storage-related building block that is in Rust. The goal is to use it to replace revlog, obsstore and packfiles. Extern crates that are likely useful are added to reduce future churns. Reviewed By: DurhamG Differential Revision: D7108434 fbshipit-source-id: 97ebd9ba69547d876dcecc05e604acdf9088877e
2018-03-10 05:57:11 +03:00
vlqencoding = { path = "../vlqencoding" }
[dev-dependencies]
indexedlog: use dev-logger Summary: This makes `RUST_LOG` work for indexedlog tests. Reviewed By: xavierd Differential Revision: D20286515 fbshipit-source-id: ff4a1476eb01a9067dabe3622fd598f65fe86a18
2020-03-11 00:12:46 +03:00
dev-logger = { path = "../dev-logger" }
indexedlog: use minibench to do benchmark Summary: See the previous minibench diff for the motivation. "failure" was removed from build dependencies since it's not used yet. Run benchmark a few times. It seems the first several items are less stable due to possibly warming up issues. Otherwise the result looks good enough. The test also compiles and runs much faster. ``` base16 iterating 1M bytes 0.921 ms index insertion 4.804 ms index flush 5.104 ms index lookup (memory) 2.929 ms index lookup (disk) 1.767 ms index clone (memory) 2.036 ms index clone (disk) 0.010 ms base16 iterating 1M bytes 0.853 ms index insertion 4.512 ms index flush 4.717 ms index lookup (memory) 2.907 ms index lookup (disk) 1.755 ms index clone (memory) 1.856 ms index clone (disk) 0.010 ms base16 iterating 1M bytes 1.525 ms index insertion 4.577 ms index flush 4.901 ms index lookup (memory) 2.800 ms index lookup (disk) 1.790 ms index clone (memory) 1.794 ms index clone (disk) 0.010 ms base16 iterating 1M bytes 0.768 ms index insertion 4.486 ms index flush 4.918 ms index lookup (memory) 2.658 ms index lookup (disk) 1.721 ms index clone (memory) 1.763 ms index clone (disk) 0.010 ms base16 iterating 1M bytes 0.732 ms index insertion 4.489 ms index flush 4.792 ms index lookup (memory) 2.689 ms index lookup (disk) 1.739 ms index clone (memory) 1.850 ms index clone (disk) 0.009 ms base16 iterating 1M bytes 1.124 ms index insertion 7.188 ms index flush 4.888 ms index lookup (memory) 2.829 ms index lookup (disk) 1.609 ms index clone (memory) 2.642 ms index clone (disk) 0.010 ms base16 iterating 1M bytes 1.055 ms index insertion 4.683 ms index flush 4.996 ms index lookup (memory) 2.782 ms index lookup (disk) 1.710 ms index clone (memory) 1.802 ms index clone (disk) 0.009 ms ``` Reviewed By: DurhamG Differential Revision: D7440249 fbshipit-source-id: 0f946ab184455acd40c5a38cf46ff94d9e3755c8
2018-04-02 19:47:29 +03:00
minibench = { path = "../minibench" }
tp2/rust: update rust-crates-io Summary: The big change here is updating to quickcheck 0.9, and removing the local change to re-export `rand`. This also updates all the `rand*` crates to their current versions, which match Quickcheck's API requirements. There was quite a lot of API changes, including ascii's loss of quickcheck support. Reviewed By: dtolnay Differential Revision: D18198542 fbshipit-source-id: 31957eb8aad1d1f81c8216e8a80c2712007415c7
2019-10-31 19:38:54 +03:00
quickcheck = "0.9"
indexedlog: fix benchmarks Summary: The `rand` APIs had changed. Update the benchmark code accordingly. Reviewed By: sfilipco Differential Revision: D19320122 fbshipit-source-id: 44c376b17afea52ecc548d79b44cbb50f7e9e567
2020-01-09 05:09:16 +03:00
rand_chacha = "0.2"
indexedlog: initial boilerplate Summary: First step of a storage-related building block that is in Rust. The goal is to use it to replace revlog, obsstore and packfiles. Extern crates that are likely useful are added to reduce future churns. Reviewed By: DurhamG Differential Revision: D7108434 fbshipit-source-id: 97ebd9ba69547d876dcecc05e604acdf9088877e
2018-03-10 05:57:11 +03:00
[[bench]]
indexedlog: separate benchmarks Summary: The minibench framework does not provide benchmark filtering. So let's separate benchmarks using different entry points. Reviewed By: DurhamG Differential Revision: D7440250 fbshipit-source-id: 11e7790a5074ebf4c08e33c312a490a66a921926
2018-04-02 19:47:34 +03:00
name = "index"
harness = false
[[bench]]
name = "misc"
indexedlog: initial boilerplate Summary: First step of a storage-related building block that is in Rust. The goal is to use it to replace revlog, obsstore and packfiles. Extern crates that are likely useful are added to reduce future churns. Reviewed By: DurhamG Differential Revision: D7108434 fbshipit-source-id: 97ebd9ba69547d876dcecc05e604acdf9088877e
2018-03-10 05:57:11 +03:00
harness = false
indexedlog: add benchmark for "log" Summary: Some simple benchmark for "log". The initial result running from my devserver looks like: log insertion 33.146 ms log insertion with index 106.449 ms log flush 9.623 ms log iteration (memory) 10.644 ms log iteration (disk) 11.517 ms 13.75s user 3.61s system 97% cpu 17.778 total log insertion 27.906 ms log insertion with index 107.683 ms log flush 19.204 ms log iteration (memory) 10.239 ms log iteration (disk) 11.118 ms 12.89s user 3.55s system 97% cpu 16.924 total log insertion 31.645 ms log insertion with index 109.403 ms log flush 9.416 ms log iteration (memory) 10.226 ms log iteration (disk) 10.757 ms 13.07s user 3.02s system 97% cpu 16.423 total log insertion 31.848 ms log insertion with index 109.332 ms log flush 18.345 ms log iteration (memory) 10.709 ms log iteration (disk) 11.346 ms 13.12s user 3.70s system 97% cpu 17.276 total log insertion 29.665 ms log insertion with index 106.041 ms log flush 16.159 ms log iteration (memory) 10.367 ms log iteration (disk) 11.110 ms 12.99s user 3.27s system 97% cpu 16.717 total Reviewed By: markbt Differential Revision: D13015464 fbshipit-source-id: 035fee6c8b6d0bea4cfe194eed3d58ba4b5ebcb8
2018-12-07 01:52:31 +03:00
[[bench]]
name = "log"
harness = false
Reference in New Issue Copy Permalink