sapling/eden/mononoke/cmds/backfill_derived_data
Stanislau Hlebik ffd60c01b0 blobimport: do not use batch_derive
Summary:
batch_derive() is a dangerous function to use. I'd love to delete it but this
function is very useful for backfilling, so unfortunately I can't.

The problem arises when one tries to backfill blame and unodes simultaneously
(or just derive blame  which in turn derives unodes). While batch_derive()
tries to be careful with inserting "outer" derived data's mappings (i.e. blame
mapping), it doesn't do it for inner derived data mappings (i.e. unodes). So we
might end up in the situation where we insert unodes mapping before we inserted
all the manifests for it. If this thing fails in the middle of derivation then
we have a corruption.

Let's do not use it in blobimport. It will make derivation slower, but I'd
rather make it slower than incorrect.

Reviewed By: farnz

Differential Revision: D21905619

fbshipit-source-id: c0227df195a8cf4482b2452ca928acbc5750b3e5
2020-06-10 19:29:31 -07:00
..
dry_run.rs mononoke: small refactoring before introducing Cleaner for unodes 2020-04-24 10:52:58 -07:00
main.rs blobimport: do not use batch_derive 2020-06-10 19:29:31 -07:00
warmup.rs mononoke: split warmup from backfill_derived_data 2020-04-23 00:16:30 -07:00