sapling/eden/mononoke/derived_data/fsnodes
Thomas Orozco f1e4f62e2d mononoke/fsnodes: expose FsnodeFile as the LeafId
Summary:
Fsnodes have a lot of data about files, but right now we can't access it
through a Fsnode lookup or a manifest walk, because the LeafId for a Fsnode is
just the content id and the file type.

This is a bit sad, because it means we e.g. cannot dump a manifest with file
sizes (D23471561 (179e4eb80e)).

Just changing the LeafId is easy, but that brings a new problem with Fsnode
derivation.

Indeed, deriving manifests normally expects us to have the "derive leaf"
function produce a LeafId (so we'd want to produce a `FsnodeFile`), but in
Fsnodes, this currently happens in deriving trees instead.

Unfortunately, we cannot easily just move the code that produces `FsnodeFile`
from the tree derivation to the leaf derivation, that is, do:

```
fn check_fsnode_leaf(
    leaf_info: LeafInfo<FsnodeFile, (ContentId, FileType)>,
) -> impl Future<Item = (Option<FsnodeSummary>, FsnodeFile), Error = Error>
```

Indeed, the performance of Fsnode derivation relies on all the leaves for a
given tree being derived together with the tree and its parents in context.

So, we'd need the ability for deriving a new leaf to return something different
from the actual leaf id. This means we want to return a `(ContentId,
FileType)`, even though our `LeafId` is a `FsnodeFile`.

To do this, this diff introduces a new `IntermediateLeafId` type in the
derivation. This represents the type of the leaf that is passed from deriving a
leaf to deriving a tree. We need to be able to turn a real `LeafId` into it,
because sometimes we don't re-derive leaves.

I think we could also refactor some of the code that passes a context here to
just do this through the `IntermediateLeafId`, but I didn't look into this too
much.

So, this diff does that, and uses it in Mononoke Admin so we can print file
sizes.

Reviewed By: StanislavGlebik

Differential Revision: D23497754

fbshipit-source-id: 2fc480be0b1e4d3d261da1d4d3dcd9c7b8501b9b
2020-09-04 06:30:18 -07:00
..
batch.rs Switch Loadable and Storable interfaces to new-style futures 2020-06-25 08:45:37 -07:00
Cargo.toml Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
derive.rs mononoke/fsnodes: expose FsnodeFile as the LeafId 2020-09-04 06:30:18 -07:00
lib.rs mononoke: add derive_fsnode_in_batch 2020-04-20 06:57:40 -07:00
mapping.rs Back out "mononoke: use batch_derive method in derived data utils" 2020-07-20 03:18:55 -07:00