facebook/sapling
1
1
Fork 0
mirror of https://github.com/facebook/sapling.git synced 2024-10-09 16:31:02 +03:00
Code Issues Projects Releases Wiki Activity
1d63473120
sapling/lib
History
Jun Wu 5e828307f4 indexedlog: verify checksum for all reads 
Summary:
It further slows down lookups, even when checksum is disabled, since even a
`is_none()` check is not free:

  index insertion                 4.697 ms
  index flush                     3.764 ms
  index lookup (memory)           2.878 ms
  index lookup (disk, no verify)  3.564 ms
  index lookup (disk, verified)   7.788 ms

The "verified" version basically needs 2x time due to more memory lookups.

Unfortunately this means eventual lookup performance will be slower than
gdbm, but insertion is still much faster. And the index still has a better
locking properties (lock-free read) that gdbm does not have.

With correct time complexity (no O(len(changelog)) index-only operations for
example), I'd expect it's rare for the overall performance to be bounded by
index performance. Data integrity is more important.

With a larger number of nodes, ex. 2M 20-byte strings: inserting to memory
takes 1.4 seconds, flushing to disk takes 0.9 seconds, looking up without
checksum takes 0.9 seconds, looking up with checksum takes 1.7 seconds.

Reviewed By: DurhamG

Differential Revision: D7440248

fbshipit-source-id: 020e5204606f9f0a4f68843a491009a6a6f75751
2018-04-13 21:51:42 -07:00
..
cdatapack hg: some portability fixes to py-cdatapack.h 2018-04-13 21:51:24 -07:00
clib hg: start using imported mman-win32 in the portability headers 2018-04-13 21:51:10 -07:00
indexedlog indexedlog: verify checksum for all reads 2018-04-13 21:51:42 -07:00
linelog hg: basic support for building hg using buck 2018-04-13 21:50:58 -07:00
minibench minibench: add a simple library to do benchmark 2018-04-13 21:51:42 -07:00
pathencoding pathencoding: utility for converting between bytes and paths 2018-04-13 21:51:35 -07:00
pathmatcher pathmatcher: initial Rust matcher that handles gitignore lazily 2018-04-13 21:51:40 -07:00
radixbuf radixbuf: use criterion for benchmark 2018-04-13 21:51:40 -07:00
third-party xdiff: add a preprocessing step that trims files 2018-04-13 21:51:25 -07:00
vlqencoding vlqencoding: add read_vlq_at API that works for AsRef<[u8]> 2018-04-13 21:51:19 -07:00
README.md READMEs: tweaks based on feedback 2018-01-12 12:35:52 -08:00

README.md

lib

Any native code (C/C++/Rust) that Mercurial (either core or extensions) depends on should go here. Python code, or native code that depends on Python code (e.g. #include <Python.h> or use cpython) is disallowed.

As we start to convert more of Mercurial into Rust, and write new paths entrirely in native code, we'll want to limit our dependency on Python, which is why this barrier exists.

See also hgext/extlib/README.md, mercurial/cext/README.mb.

How do I choose between lib and extlib (and cext)?

If your code is native and doesn't depend on Python (awesome!), it goes here.

Otherwise, put it in hgext/extlib (if it's only used by extensions) or mercurial/cext (if it's used by extensions or core).