Summary:
Add --derived-data-type=fsnodes to blobimport to a couple of walker tests so we have test data present to load.
Includes a small change to library.sh to add default_setup_pre_blobimport entry point used by these tests
Reviewed By: StanislavGlebik
Differential Revision: D21202480
fbshipit-source-id: d7eb3e5736531a11da87d92d0d03a528ff2c91a7
Summary: Make it possible to traverse fsnodes in walker.
Reviewed By: ahornby
Differential Revision: D21153883
fbshipit-source-id: 047ab73466f48048a34cb52e7e0f6d04cda3143b
Summary:
Allow us to see the sizes for each node type (e.g. manifests, bonsais etc), and extends the default reporting to all types.
The progress.rs changes update its summary by type reporting to be reusable, and then it is reused by the changes to sizing.rs.
Reviewed By: krallin
Differential Revision: D20560962
fbshipit-source-id: f09b45b34f42c5178ba107dd155abf950cd090a7
Summary:
Use the new sampling blobstore and sampling key in existing compression-benefit subcommand and check the new vs old reported sizes.
The overall idea for these changes is that the walker uses a CoreContext tagged with a SamplingKey to correlate walker steps for a node to the underlying blobstore reads, this allows us to track overall bytes size (used in scrub stats) or the data itself (used in compression benefit) per node type.
The SamplingVisitor and NodeSamplingHandler cooperate to gather the sampled data into the maps in NodeSamplingHandler, which the output stream from the walk then operates on, e.g. to compress the blobs and report on compression benefit.
The main new logic sits in sampling.rs, it is used from sizing.rs (and later in stack from scrub.rs)
Reviewed By: krallin
Differential Revision: D20534841
fbshipit-source-id: b20e10fcefa5c83559bdb15b86afba209c63119a
