facebook/sapling
1
1
Fork 0
mirror of https://github.com/facebook/sapling.git synced 2024-10-10 08:47:12 +03:00
Code Issues Projects Releases Wiki Activity

mononoke: add BonsaiHgMappingToHgBonsaiMapping edge to assist LFS validation

Browse Source 
Summary:
The existing query to establish HgChangesetId on the path to FileContentMetadata for LFS validation is quite complex, using HgFilenode linknodes.

This change adds an optional edge from BonsaiHgMappingToHgBonsaiMapping that can be used to simplify the LFS validation case and load less data to get there.

Reviewed By: mitrandir77

Differential Revision: D26975799

fbshipit-source-id: 799acb8228721c1878f33254ebfa5e6345673e5d
This commit is contained in:
Alex Hornby 2021-03-12 02:52:51 -08:00 committed by Facebook GitHub Bot
parent 40bc3dec6b
commit 0035efe1f5
3 changed files with 32 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
eden/mononoke/tests/integration/test-walker-validate-lfs.t
View File

@ -60,3 +60,26 @@ Check scuba data is logged for lfs and that it contains useful hg changeset and
0,"file_content_is_lfs","content.blake2.896ad5879a5df0403bfc93fc96507ad9c93b31b11f3d0fa05445da7918241e5d","C","FileContentMetadata","repo","Changeset","hgchangeset.sha1.26805aba1e600a82e93661149f2313866a221a7b","HgBonsaiMapping","validate",
0,"file_content_is_lfs","content.blake2.e164fd53a3714f754d5f5763688bea02d99123436e51e9ed9c85ad04fdc52222","foo/bar","FileContentMetadata","repo","Changeset","hgchangeset.sha1.d9a770e9f783614a05c7c060830b08e65ce7d774","HgBonsaiMapping","validate",
0,"file_content_is_lfs","content.blake2.eb56488e97bb4cf5eb17f05357b80108a4a71f6c3bab52dfcaec07161d105ec9","A","FileContentMetadata","repo","Changeset","hgchangeset.sha1.426bada5c67598ca65036d57d9e4b64b0c1ce7a0","HgBonsaiMapping","validate",
validate with LFS enabled, deep with simpler query. Should have same output but touch less nodes to get there.
$ mononoke_walker validate --include-check-type=FileContentIsLfs --scuba-log-file scuba-validate-deep2.json -I deep -I BonsaiHgMappingToHgBonsaiMapping -X BonsaiHgMappingToHgChangeset -X ChangesetToBonsaiParent -X ChangesetToBonsaiHgMapping -i bonsai -i FileContent -i FileContentMetadata -i HgBonsaiMapping -i BonsaiHgMapping -q -p BonsaiHgMapping 2>&1 | strip_glog
Walking edge types [BonsaiHgMappingToHgBonsaiMapping, ChangesetToFileContent, FileContentToFileContentMetadata, HgBonsaiMappingToChangeset]
Walking node types [BonsaiHgMapping, Changeset, FileContent, FileContentMetadata, HgBonsaiMapping]
Performing check types [FileContentIsLfs]
Repo bounds: (1, 5)
Starting chunk 1 with bounds (1, 5)
Seen,Loaded: * (glob)
Walked* (glob)
Nodes,Pass,Fail:20,4,0; EdgesChecked:4; CheckType:Pass,Fail Total:4,0 FileContentIsLfs:4,0
Deferred: 0
Completed in 1 chunks of size 100000
Check scuba data is logged for lfs and that it contains useful hg changeset and path in via_node_key and node_path
$ wc -l < scuba-validate-deep2.json
4
$ jq -r '.int * .normal | [ .check_fail, .check_type, .node_key, .node_path, .node_type, .repo, .src_node_type, .via_node_key, .via_node_type, .walk_type, .error_msg ] | @csv' < scuba-validate-deep2.json | sort
0,"file_content_is_lfs","content.blake2.55662471e2a28db8257939b2f9a2d24e65b46a758bac12914a58f17dcde6905f","B","FileContentMetadata","repo","Changeset","hgchangeset.sha1.112478962961147124edd43549aedd1a335e44bf","HgBonsaiMapping","validate",
0,"file_content_is_lfs","content.blake2.896ad5879a5df0403bfc93fc96507ad9c93b31b11f3d0fa05445da7918241e5d","C","FileContentMetadata","repo","Changeset","hgchangeset.sha1.26805aba1e600a82e93661149f2313866a221a7b","HgBonsaiMapping","validate",
0,"file_content_is_lfs","content.blake2.e164fd53a3714f754d5f5763688bea02d99123436e51e9ed9c85ad04fdc52222","foo/bar","FileContentMetadata","repo","Changeset","hgchangeset.sha1.d9a770e9f783614a05c7c060830b08e65ce7d774","HgBonsaiMapping","validate",
0,"file_content_is_lfs","content.blake2.eb56488e97bb4cf5eb17f05357b80108a4a71f6c3bab52dfcaec07161d105ec9","A","FileContentMetadata","repo","Changeset","hgchangeset.sha1.426bada5c67598ca65036d57d9e4b64b0c1ce7a0","HgBonsaiMapping","validate",

2
eden/mononoke/walker/src/graph.rs
View File

@ -338,7 +338,7 @@ create_graph!(
UnodeMapping
]
),
(BonsaiHgMapping, ChangesetKey<ChangesetId>, [HgChangeset]),
(BonsaiHgMapping, ChangesetKey<ChangesetId>, [HgBonsaiMapping, HgChangeset]),
(PhaseMapping, ChangesetId, []),
(
PublishedBookmarks,

8
eden/mononoke/walker/src/walk.rs
View File

@ -651,6 +651,14 @@ async fn bonsai_to_hg_mapping_step<'a, V: 'a + VisitOne>(
Some(hg_cs_id) => {
let hg_cs_id = hg_cs_id.0;
let mut edges = vec![];
// This seems like a nonsense edge, but its a way to establish HgChangesetId on the way to Bonsai Changeset
// which is useful in LFS validation. The edge is disabled by default.
checker.add_edge(&mut edges, EdgeType::BonsaiHgMappingToHgBonsaiMapping, || {
Node::HgBonsaiMapping(ChangesetKey {
inner: hg_cs_id,
filenode_known_derived,
})
});
checker.add_edge(&mut edges, EdgeType::BonsaiHgMappingToHgChangeset, || {
Node::HgChangeset(ChangesetKey {
inner: hg_cs_id,