rename diskMaterialized

Summary:
In the fsck code, diskMaterialized does not line up with EdenFS's internal
definition of materialized. The "materialized" in diskMaterialized here is used
to mean the contents of the file are present on disk where disk means prjfs.
EdenFS may or may not think that the inode is "materialized".

Particularly, if the file is renamed EdenFS will think that the inode is
materialized, but it would not be "diskMaterialized".

This confused me and I think we should avoid the "materialized" terminology (and
probably the "disk" terminology too because materialized inodes can be "on disk"
in the overlay).

I propose the admittedly more verbose, but clear "contentsPresentInPrjfs".

Reviewed By: xavierd

Differential Revision: D41758686

fbshipit-source-id: ddd3b6eb78a9b233747c6923f01d7832cc3072ff
This commit is contained in:
Katie Mancini 2023-02-06 16:55:36 -08:00 committed by Facebook GitHub Bot
parent 7401b22f5c
commit 2fc72e0e72
2 changed files with 46 additions and 19 deletions

View File

@ -109,6 +109,30 @@ contents can always be fetched from the source control repository.
For more details see the
[Inode Materialization](Inodes.md#inode-materialization) documentation.
In ProjFS mounts, there is an additional special case for materialized files.
Files that have been renamed are considered materialized inodes. Technically,
we still know the source control object associated with the inode, however,
we no longer store this association in the overlay. ProjFS always will
make a read request for these files with the original path and reads are
only served from source control objects in ProjFS.
### Populated
Inodes or files are considered "populated" when their contents have been
observed by the kernel, but the file has not yet been modified.
For ProjFS mounts "populated" this means the contents are present on
the filesystem and reads are going to be directly handled by ProjFS
until we invalidate the file. In FUSE and NFS mounts this means the kernel
may have the file contents in its caches, though FUSE or NFS may have
decided to evict them from cache of its own will.
For ProjFS mounts, "populated" is roughly equivalent to the hydrated
placeholder state for files. For directories "populated" is roughly equivalent
to placeholders without materialized children.
No populated files and directories correspond to materialized inodes and
vice versa. These states are intentionally independent. i.e. populated
is defined as files the kernel knows about - materialized ones.
### Overlay
The overlay is where EdenFS stores information about

View File

@ -247,10 +247,11 @@ void removeOverlayEntry(
struct FsckFileState {
bool onDisk = false;
// diskMaterialized is true if:
// - a file is full
// - a directory is full or a descendant is materialized or tombstoned.
bool diskMaterialized = false;
// populatedOrFullOrTomb is true if:
// - a file is full, hydrated or tomstoned
// - a directory is full or a dirty placeholder or a descendant is
// populatedOrFullOrTomb
bool populatedOrFullOrTomb = false;
// diskEmptyPlaceholder is true if:
// - a file is virtual or a placeholder
// - a directory is a placeholder and has no children (placeholder or
@ -316,7 +317,7 @@ void populateDiskState(
// On Windows, EdenFS consider all special files (symlinks, sockets, etc)
// to be regular.
state.diskDtype = dtype_t::Regular;
state.diskMaterialized = true;
state.populatedOrFullOrTomb = true;
return;
}
@ -356,11 +357,11 @@ void populateDiskState(
state.diskDtype = dtype;
state.diskTombstone = detectedTombstone;
// It can also be diskMaterialized if a descendant directory is
// It can also be populated if a descendant directory is
// materialized. But that is checked later when processing the children.
state.diskMaterialized = detectedFull || detectedTombstone;
state.populatedOrFullOrTomb = detectedFull || detectedTombstone;
// It's an empty placeholder unless it's materialized or it has children.
state.diskEmptyPlaceholder = !state.diskMaterialized;
state.diskEmptyPlaceholder = !state.populatedOrFullOrTomb;
if (dtype == dtype_t::Dir && !directoryIsEmpty(wPath.c_str())) {
state.diskEmptyPlaceholder = false;
@ -437,7 +438,7 @@ std::optional<InodeNumber> fixup(
} else if (state.diskTombstone) {
// state.shouldExist defaults to false
} else { // if file exists normally on disk
if (!state.inScm && !state.diskMaterialized) {
if (!state.inScm && !state.populatedOrFullOrTomb) {
// Stop fixing this up since we can't materialize if it's not in scm.
// TODO: This is likely caused by EdenFS not having called PrjDeleteFile
// in a previous checkout operation. We should probably call it here or
@ -447,7 +448,8 @@ std::optional<InodeNumber> fixup(
return std::nullopt;
} else {
state.desiredDtype = state.diskDtype;
state.desiredHash = state.diskMaterialized ? std::nullopt : state.scmHash;
state.desiredHash =
state.populatedOrFullOrTomb ? std::nullopt : state.scmHash;
state.shouldExist = true;
}
}
@ -460,7 +462,7 @@ std::optional<InodeNumber> fixup(
state.inOverlay,
state.inScm,
state.diskTombstone,
state.diskMaterialized);
state.populatedOrFullOrTomb);
if (state.shouldExist) {
bool out_of_sync = !state.inOverlay ||
@ -592,7 +594,7 @@ bool processChildren(
}
// Recurse for any children.
bool anyChildMaterialized = false;
bool anyChildPopulatedOrFullOrTomb = false;
for (auto& [childName, childState] : children) {
auto childPath = path + childName;
XLOGF(DBG9, "process child - {}", childPath);
@ -604,7 +606,7 @@ bool processChildren(
inodeNumber,
insensitiveOverlayDir);
anyChildMaterialized |= childState.diskMaterialized;
anyChildPopulatedOrFullOrTomb |= childState.populatedOrFullOrTomb;
if (childState.desiredDtype == dtype_t::Dir && childState.onDisk &&
!childState.diskEmptyPlaceholder && childInodeNumberOpt.has_value()) {
@ -623,8 +625,8 @@ bool processChildren(
auto childInodeNumber = *childInodeNumberOpt;
auto childOverlayDir = *inodeCatalog.loadOverlayDir(childInodeNumber);
auto childInsensitiveOverlayDir = toPathMap(childOverlayDir);
bool childMaterialized = childState.diskMaterialized;
childMaterialized |= processChildren(
bool childPopulatedOrFullOrTomb = childState.populatedOrFullOrTomb;
childPopulatedOrFullOrTomb |= processChildren(
inodeCatalog,
childPath,
root,
@ -634,14 +636,15 @@ bool processChildren(
callback,
logFrequency,
traversedDirectories);
anyChildMaterialized |= childMaterialized;
anyChildPopulatedOrFullOrTomb |= childPopulatedOrFullOrTomb;
if (childMaterialized && childState.desiredHash != std::nullopt) {
if (childPopulatedOrFullOrTomb &&
childState.desiredHash != std::nullopt) {
XLOGF(
DBG9,
"Directory {} has a materialized child, and therefore is materialized too. Marking.",
childPath);
childState.diskMaterialized = true;
childState.populatedOrFullOrTomb = true;
childState.desiredHash = std::nullopt;
// Refresh the parent state so we see and update the current overlay
// entry.
@ -659,7 +662,7 @@ bool processChildren(
}
}
return anyChildMaterialized;
return anyChildPopulatedOrFullOrTomb;
}
void scanCurrentDir(