facebook/sapling
1
1
Fork 0
mirror of https://github.com/facebook/sapling.git synced 2024-10-10 08:47:12 +03:00
Code Issues Projects Releases Wiki Activity

Switch to skiplist with shorter jumps

Browse Source 
Summary:
Pointing the configs to a new package allows us to control the rollout more easily than overriding the existing skiplist in manifold.

This skiplist features a single skip entry for each node with a depth of up to 512 commits. Previously depth was 32k. This means on average we will have to do 16k iterations to find the commit we want after doing the last skiplist jump.

If we were to jump to the very beginning of the repo that would mean before we had to do ~130 skips and then on average 16k steps. With this skiplist we would do ~8k skips and 256 steps. As skips and steps are equally expensive this should be faster in almost every scenario. The total size of the skiplist is not affected by this change (as each node still contains one skip entry) but the new skiplist also covers the complete history, not only the 2kk most recent commits. This leads to size increase of ~20%.

Reviewed By: StanislavGlebik, HarveyHunt

Differential Revision: D14540318

fbshipit-source-id: 86fda5b6c57a06bc4a77e30625014ec119e7a155
This commit is contained in:
David Budischek 2019-03-22 05:28:57 -07:00 committed by Facebook Github Bot
parent de3f6eebe8
commit 9579288aaf
1 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
cmds/admin/main.rs
View File

@ -575,8 +575,10 @@ fn build_skiplist_index<S: ToString>(
sql_changesets: SqlChangesets,
) -> BoxFuture<(), Error> {
let blobstore = repo.get_blobstore();
let skiplist_depth = 16;
let max_index_depth = 2000000000;
// skiplist will jump up to 2^9 changesets
let skiplist_depth = 10;
// Index all changesets
let max_index_depth = 20000000000;
let skiplist_index = SkiplistIndex::with_skip_edge_count(skiplist_depth);
let key = key.to_string();