sapling/eden/mononoke/derived_data
Thomas Orozco f1e4f62e2d mononoke/fsnodes: expose FsnodeFile as the LeafId
Summary:
Fsnodes have a lot of data about files, but right now we can't access it
through a Fsnode lookup or a manifest walk, because the LeafId for a Fsnode is
just the content id and the file type.

This is a bit sad, because it means we e.g. cannot dump a manifest with file
sizes (D23471561 (179e4eb80e)).

Just changing the LeafId is easy, but that brings a new problem with Fsnode
derivation.

Indeed, deriving manifests normally expects us to have the "derive leaf"
function produce a LeafId (so we'd want to produce a `FsnodeFile`), but in
Fsnodes, this currently happens in deriving trees instead.

Unfortunately, we cannot easily just move the code that produces `FsnodeFile`
from the tree derivation to the leaf derivation, that is, do:

```
fn check_fsnode_leaf(
    leaf_info: LeafInfo<FsnodeFile, (ContentId, FileType)>,
) -> impl Future<Item = (Option<FsnodeSummary>, FsnodeFile), Error = Error>
```

Indeed, the performance of Fsnode derivation relies on all the leaves for a
given tree being derived together with the tree and its parents in context.

So, we'd need the ability for deriving a new leaf to return something different
from the actual leaf id. This means we want to return a `(ContentId,
FileType)`, even though our `LeafId` is a `FsnodeFile`.

To do this, this diff introduces a new `IntermediateLeafId` type in the
derivation. This represents the type of the leaf that is passed from deriving a
leaf to deriving a tree. We need to be able to turn a real `LeafId` into it,
because sometimes we don't re-derive leaves.

I think we could also refactor some of the code that passes a context here to
just do this through the `IntermediateLeafId`, but I didn't look into this too
much.

So, this diff does that, and uses it in Mononoke Admin so we can print file
sizes.

Reviewed By: StanislavGlebik

Differential Revision: D23497754

fbshipit-source-id: 2fc480be0b1e4d3d261da1d4d3dcd9c7b8501b9b
2020-09-04 06:30:18 -07:00
..
blame Back out "mononoke: use batch_derive method in derived data utils" 2020-07-20 03:18:55 -07:00
changeset_info Remove crate_root from Rust dependency info 2020-08-31 14:43:24 -07:00
deleted_files_manifest better public api for deleted files manifest 2020-07-23 07:34:51 -07:00
fastlog don't error out when path doesn't exist 2020-07-30 03:25:01 -07:00
filenodes Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
fsnodes mononoke/fsnodes: expose FsnodeFile as the LeafId 2020-09-04 06:30:18 -07:00
mercurial_derived_data BlobRepoHg extension trait. 2020-06-22 07:29:19 -07:00
src Back out "mononoke: use batch_derive method in derived data utils" 2020-07-20 03:18:55 -07:00
unodes Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
utils Back out "mononoke: use batch_derive method in derived data utils" 2020-07-20 03:18:55 -07:00
Cargo.toml Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00