sapling/eden
Andrey Chursin c24bb6096b Make ObjectId a variable length hash [proxy hash removal 4/n]
Summary:
This diff modifies ObjectId structure to support storing more information directly in ObjectId, avoiding proxy objects like HgProxyHash and ReCasDigestProxyHash.

Storing this information directly in ObjectId allows to avoid additional db access for loading/storing those proxy hash objects, reducing file access latency in hot case.

**New ObjectId format**

ObjectId can now content variable length hash.

The variable length can be used in following cases:
(1) To replace ReCasDigestProxyHash, extra 8 bytes can be used to store size portion of `remote_execution::TDigest`
(2) To replace HgProxyHash, extra 1 byte can be used to identify whether ObjectId represents ProxyHashId or an HgId.

In the future, ObjectId can contain path or other information needed to implement ACL

**Compatibility notes**

For compatibility reasons, we only currently initialize ObjectId with 20-bytes content.

This is essential to allow smooth migration to a new format
(a) Until new hash format is actually used(e.g. we switch HgBackingStore to store HgId inside ObjectId), this code is backwards compatible with other EdenFs versions. This revision can be safely rolled back and previous version of EdenFs can still read inode data written by this version
(b) When we introduce support for new ObjectId into HgBackingStore, we can gate it's usage behind config, allowing controlled slow roll out of new ObjectId format.

**ToDo**

Just to track progress of removing proxy hashes, few things are still left:
* We need different format for SerializedTreeMetadata
* We need to support new format for thriftHash
* We need to actually switch HgBackingStore to embed HgId information into ObjectId, instead of using ProxyHash

Not planned:
* Migration of ReCasDigestProxyHash - I don't know how to test it, so someone else should probably do that

Reviewed By: chadaustin

Differential Revision: D31668130

fbshipit-source-id: 720127354c648651bb35e850beb8dd252a5566b2
2021-10-22 17:52:01 -07:00
..
fs Make ObjectId a variable length hash [proxy hash removal 4/n] 2021-10-22 17:52:01 -07:00
hg-server Remove imports of prelude items 2021-10-21 20:06:41 -07:00
integration suppress errors in fbcode/eden - batch 1 2021-10-12 13:08:00 -07:00
locale
mononoke mononoke: make it possible to split stack by copy info 2021-10-22 13:37:35 -07:00
scm amend: prefetch file history data 2021-10-22 17:13:04 -07:00
test_support test_support: canonicalize the temporary directory path 2021-08-16 16:08:45 -07:00
test-data fix fsck snapshot integration tests 2021-07-14 16:20:04 -07:00
.gitignore
Eden.project.toml