sapling/eden/fs/store
Wez Furlong 78ef6207e3 use rocksdb column families to separate key spaces
Summary:
We're running down a performance problem with hg status that we believe
is something happening at a higher level in the code but noticed that there
were a lot of reads of the rocksdb SST files.  In the strace output for those
we observed file content data being read.  The status operation shouldn't
need file contents; what's happening is that we're over-fetching some metadata
but happen to be scooping up the file contents from the SST file because we
use the same key prefixes and differentiate the keyspace with key suffixes.

This diff implements the use of rocksdb column families to partition things
more effectively and results in a speed up of around 6x in this scenario.

Furthermore, applying point lookup optimization options yields an additional
2x performance improvement to our rocksdb performance.

As part of this diff, I've removed the hash set that we were using to allow
checking whether a key was present in the store; it wasn't very useful
and would have had to be split into one set per keyspace with this diff;
easier to just remove it.

Reviewed By: bolinfest

Differential Revision: D5781906

fbshipit-source-id: 97f068ade546fd09f391e60a7a57fec0e9081e67
2017-09-07 14:50:42 -07:00
..
git Remove no-longer-needed @manual for rocksdb 2017-09-01 19:31:46 -07:00
hg use rocksdb column families to separate key spaces 2017-09-07 14:50:42 -07:00
test use rocksdb column families to separate key spaces 2017-09-07 14:50:42 -07:00
BackingStore.h update all copyright statements to "2016-present" 2017-01-20 22:03:02 -08:00
BlobMetadata.h update all copyright statements to "2016-present" 2017-01-20 22:03:02 -08:00
EmptyBackingStore.cpp update all copyright statements to "2016-present" 2017-01-20 22:03:02 -08:00
EmptyBackingStore.h apply clang-tidy modernize-use-override 2017-05-23 16:10:26 -07:00
IObjectStore.h drop Future from the name of ObjectStore APIs 2017-06-21 17:20:50 -07:00
LocalStore.cpp use rocksdb column families to separate key spaces 2017-09-07 14:50:42 -07:00
LocalStore.h use rocksdb column families to separate key spaces 2017-09-07 14:50:42 -07:00
ObjectStore.cpp update logging statements to use folly logging APIs 2017-06-22 13:50:13 -07:00
ObjectStore.h drop Future from the name of ObjectStore APIs 2017-06-21 17:20:50 -07:00
StoreResult.cpp update all copyright statements to "2016-present" 2017-01-20 22:03:02 -08:00
StoreResult.h update all copyright statements to "2016-present" 2017-01-20 22:03:02 -08:00
TARGETS allow specifying --config options to hg_import_helper.py 2017-08-02 17:07:19 -07:00