sapling/eden/mononoke/blobimport_lib/src
Thomas Orozco 9493a05e7b mononoke/filestore: update store_bytes to chunk content
Summary:
This updates the store_bytes method to chunk incoming data instead of uploading
it as-is. This is unfortunately a bit hacky (but so was the previous
implementation), since it means we have to hash the data before it has gone
through the Filestore's preparation.

That said, one of the invariants of the filestore is that chunk size shouldn't
affect the Content ID (and there is fairly extensive test coverage for this),
so, notionally, this does work.

Performance-wise, it does mean we are hashing the object twice. That actually
was the case before as well anyway (since obtain the ContentId for FileContents
would clone them then hash them).

The upshot of this change is that large files uploaded through unbundle will
actually be chunked (whereas before, they wouldn't be).

Long-term, we should try and delete this method, as it is quite unsavory to
begin with. But, for now, we don't really have a choice since our content
upload path does rely on its existence.

Reviewed By: StanislavGlebik

Differential Revision: D20281937

fbshipit-source-id: 78d584b2f9eea6996dd1d4acbbadc10c9049a408
2020-03-06 07:43:07 -08:00
..
bookmark.rs rust: Move futures-old rdeps to renamed futures-old 2020-03-02 21:02:50 -08:00
changeset.rs mononoke/filestore: update store_bytes to chunk content 2020-03-06 07:43:07 -08:00
concurrency.rs rust: Move futures-old rdeps to renamed futures-old 2020-03-02 21:02:50 -08:00
lib.rs rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00