sapling/eden/fs/inodes/TreeInode.h

262 lines
8.4 KiB
C
Raw Normal View History

/*
* Copyright (c) 2016-present, Facebook, Inc.
* All rights reserved.
*
* This source code is licensed under the BSD-style license found in the
* LICENSE file in the root directory of this source tree. An additional grant
* of patent rights can be found in the PATENTS file in the same directory.
*
*/
#pragma once
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
#include <folly/Optional.h>
#include "eden/fs/inodes/InodeBase.h"
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
#include "eden/fs/model/Hash.h"
#include "eden/utils/PathMap.h"
namespace facebook {
namespace eden {
class EdenMount;
class FileHandle;
class InodeMap;
class ObjectStore;
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
class Overlay;
class RenameLock;
class Tree;
// Represents a Tree instance in a form that FUSE can consume
class TreeInode : public InodeBase {
public:
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
/** Represents a directory entry.
* A directory entry holds the combined Tree and Overlay data;
* if a directory is only partially materialized the entire
* directory contents be part of this data, but the individual
* entries will indicate whether they have been materialized or not.
*/
struct Entry {
/** The complete st_mode value for this entry */
mode_t mode;
/** If !materialized, the blob or tree hash for this entry in
* the local store */
folly::Optional<Hash> hash;
/** true if the entry has been materialized to the overlay.
* For a directory this means that the directory exists, for
* a file it means that the file exists */
bool materialized{false};
introduce a new InodeMap class Summary: This diff starts adding a new InodeMap class. This class will eventually consolidate the functionality of InodeNameMap plus the inode map stored in EdenDispatcher. This new class will bring several new benefits: - All inode mapping logic consolidated into a single class, with a single lock protecting the maps. - A well-defined model for loaded vs unloaded inodes. InodeMap explicitly tracks inodes that have InodeBase objects created for them vs inodes that have an inode number allocated (for FUSE) but do not have an InodeBase object in memory. This will make it possible to unload Inode objects on demand to reduce memory usage. - Tracking of pending loads, and de-duplicating load requests. This ensures that only one Inode object ever exists for a given inode number / path. If a second request to load an Inode arrives when a previous load request is still in progress, InodeMap deals with this situation properly. - Better support for using Inode objects without FUSE. With the old code, attempts to interact with Inode objects without going through the FUSE dispatch (say, when processing a thrift call) could result in inconsistent state. New inodes created would not be put into the EdenDispatcher map, which could result in problems. - More convenient child inode access from TreeInode. With this change, the TreeInode class can easily tell which of its children are loaded. This makes it easier to do tasks which only need to operate on existing loaded inode state (for instance, dirstate computation). - Support for saving and loading state, to implement graceful restart.. InodeMap provides a central place to write out inode state on shutdown and restoring it on startup. Saved inodes can easily be restored to an "unloaded" state on startup. This code is not implemented yet as part of this diff, but it should be straightforward to add in future diffs. Reviewed By: bolinfest Differential Revision: D4318060 fbshipit-source-id: d9b16430fc8367e3516e788d1e991e5163aa6997
2016-12-23 02:34:54 +03:00
/**
* A pointer to the child inode, if it is loaded, or null if it is not
* loaded.
*
* Note that we store this as a raw pointer. Children inodes hold a
* reference to their parent TreeInode, not the other way around.
* Children inodes can be destroyed only in one of two ways:
* - Being unlinked, then having their last reference go away.
* In this case they will be removed from our entries list when they are
* unlinked.
* - Being unloaded (after their reference count is already 0). In this
* case the parent TreeInodes responsible for triggering unloading of its
* children, so it resets this pointer to null when it unloads the child.
*
* TODO: This should perhaps be a folly::Variant with the above data.
* If the child is loaded, it should be the source of truth about all of
* the data for the child.
*/
InodeBase* inode{nullptr};
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
};
/** Represents a directory in the overlay */
struct Dir {
/** The direct children of this directory */
PathMap<std::unique_ptr<Entry>> entries;
/** If the origin of this dir was a Tree, the hash of that tree */
folly::Optional<Hash> treeHash;
/** true if the dir has been materialized to the overlay.
* If the contents match the original tree, this is false. */
bool materialized{false};
};
/** Holds the results of a create operation.
*
* It is important that the file handle creation respect O_EXCL if
* it set in the flags parameter to TreeInode::create.
*/
struct CreateResult {
/// file attributes and cache ttls.
fusell::Dispatcher::Attr attr;
/// The newly created inode instance.
InodePtr inode;
/// The newly opened file handle.
std::shared_ptr<FileHandle> file;
explicit CreateResult(const fusell::MountPoint* mount) : attr(mount) {}
};
TreeInode(
fuse_ino_t ino,
TreeInodePtr parent,
PathComponentPiece name,
Entry* entry,
std::unique_ptr<Tree>&& tree);
/// Construct an inode that only has backing in the Overlay area
TreeInode(
fuse_ino_t ino,
TreeInodePtr parent,
PathComponentPiece name,
Entry* entry,
Dir&& dir);
/// Constructors for the root TreeInode
TreeInode(EdenMount* mount, std::unique_ptr<Tree>&& tree);
TreeInode(EdenMount* mount, Dir&& tree);
~TreeInode();
folly::Future<fusell::Dispatcher::Attr> getattr() override;
fusell::Dispatcher::Attr getAttrLocked(const Dir* contents);
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
/** Implements the InodeBase method used by the Dispatcher
* to create the Inode instance for a given name */
folly::Future<InodePtr> getChildByName(PathComponentPiece namepiece);
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
introduce a new InodeMap class Summary: This diff starts adding a new InodeMap class. This class will eventually consolidate the functionality of InodeNameMap plus the inode map stored in EdenDispatcher. This new class will bring several new benefits: - All inode mapping logic consolidated into a single class, with a single lock protecting the maps. - A well-defined model for loaded vs unloaded inodes. InodeMap explicitly tracks inodes that have InodeBase objects created for them vs inodes that have an inode number allocated (for FUSE) but do not have an InodeBase object in memory. This will make it possible to unload Inode objects on demand to reduce memory usage. - Tracking of pending loads, and de-duplicating load requests. This ensures that only one Inode object ever exists for a given inode number / path. If a second request to load an Inode arrives when a previous load request is still in progress, InodeMap deals with this situation properly. - Better support for using Inode objects without FUSE. With the old code, attempts to interact with Inode objects without going through the FUSE dispatch (say, when processing a thrift call) could result in inconsistent state. New inodes created would not be put into the EdenDispatcher map, which could result in problems. - More convenient child inode access from TreeInode. With this change, the TreeInode class can easily tell which of its children are loaded. This makes it easier to do tasks which only need to operate on existing loaded inode state (for instance, dirstate computation). - Support for saving and loading state, to implement graceful restart.. InodeMap provides a central place to write out inode state on shutdown and restoring it on startup. Saved inodes can easily be restored to an "unloaded" state on startup. This code is not implemented yet as part of this diff, but it should be straightforward to add in future diffs. Reviewed By: bolinfest Differential Revision: D4318060 fbshipit-source-id: d9b16430fc8367e3516e788d1e991e5163aa6997
2016-12-23 02:34:54 +03:00
/**
* Get the inode object for a child of this directory.
*
* The Inode object will be loaded if it is not already loaded.
*/
folly::Future<InodePtr> getOrLoadChild(PathComponentPiece name);
folly::Future<TreeInodePtr> getOrLoadChildTree(PathComponentPiece name);
/**
* Recursively look up a child inode.
*
* The Inode object in question, and all intervening TreeInode objects,
* will be loaded if they are not already loaded.
*/
folly::Future<InodePtr> getChildRecursive(RelativePathPiece name);
fuse_ino_t getChildInodeNumber(PathComponentPiece name);
introduce a new InodeMap class Summary: This diff starts adding a new InodeMap class. This class will eventually consolidate the functionality of InodeNameMap plus the inode map stored in EdenDispatcher. This new class will bring several new benefits: - All inode mapping logic consolidated into a single class, with a single lock protecting the maps. - A well-defined model for loaded vs unloaded inodes. InodeMap explicitly tracks inodes that have InodeBase objects created for them vs inodes that have an inode number allocated (for FUSE) but do not have an InodeBase object in memory. This will make it possible to unload Inode objects on demand to reduce memory usage. - Tracking of pending loads, and de-duplicating load requests. This ensures that only one Inode object ever exists for a given inode number / path. If a second request to load an Inode arrives when a previous load request is still in progress, InodeMap deals with this situation properly. - Better support for using Inode objects without FUSE. With the old code, attempts to interact with Inode objects without going through the FUSE dispatch (say, when processing a thrift call) could result in inconsistent state. New inodes created would not be put into the EdenDispatcher map, which could result in problems. - More convenient child inode access from TreeInode. With this change, the TreeInode class can easily tell which of its children are loaded. This makes it easier to do tasks which only need to operate on existing loaded inode state (for instance, dirstate computation). - Support for saving and loading state, to implement graceful restart.. InodeMap provides a central place to write out inode state on shutdown and restoring it on startup. Saved inodes can easily be restored to an "unloaded" state on startup. This code is not implemented yet as part of this diff, but it should be straightforward to add in future diffs. Reviewed By: bolinfest Differential Revision: D4318060 fbshipit-source-id: d9b16430fc8367e3516e788d1e991e5163aa6997
2016-12-23 02:34:54 +03:00
folly::Future<std::shared_ptr<fusell::DirHandle>> opendir(
const struct fuse_file_info& fi);
folly::Future<folly::Unit> rename(
PathComponentPiece name,
TreeInodePtr newParent,
PathComponentPiece newName);
std::unique_ptr<InodeBase> renameHelper(
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
Dir* sourceContents,
PathComponentPiece sourceName,
TreeInodePtr destParent,
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
Dir* destContents,
PathComponentPiece destName,
const RenameLock& renameLock);
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
fuse_ino_t getInode() const;
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
const folly::Synchronized<Dir>& getContents() const {
return contents_;
}
folly::Synchronized<Dir>& getContents() {
return contents_;
}
/**
* Get the InodeMap for this tree's EdenMount.
*
* The InodeMap is guaranteed to remain valid for at least the lifetime of
* the TreeInode object.
*/
InodeMap* getInodeMap() const;
/**
* Get the ObjectStore for this mount point.
*
* The ObjectStore is guaranteed to remain valid for at least the lifetime of
* the TreeInode object. (The ObjectStore is owned by the EdenMount.)
*/
ObjectStore* getStore() const;
const std::shared_ptr<Overlay>& getOverlay() const;
folly::Future<CreateResult>
create(PathComponentPiece name, mode_t mode, int flags);
folly::Future<fuse_entry_param> symlink(
PathComponentPiece name,
folly::StringPiece contents);
TreeInodePtr mkdir(PathComponentPiece name, mode_t mode);
folly::Future<folly::Unit> unlink(PathComponentPiece name);
folly::Future<folly::Unit> rmdir(PathComponentPiece name);
/** Called in a thrift context to switch the active snapshot.
* Since this is called in a thrift context, RequestData::get() won't
* return the usual results and the appropriate information must
* be passed down from the thrift server itself.
*/
void performCheckout(const Hash& hash);
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
/** Ensure that the overlay is tracking metadata for this inode
* This is required whenever we are about to make a structural change
* in the tree; renames, creation, deletion. */
void materializeDirAndParents();
introduce a new InodeMap class Summary: This diff starts adding a new InodeMap class. This class will eventually consolidate the functionality of InodeNameMap plus the inode map stored in EdenDispatcher. This new class will bring several new benefits: - All inode mapping logic consolidated into a single class, with a single lock protecting the maps. - A well-defined model for loaded vs unloaded inodes. InodeMap explicitly tracks inodes that have InodeBase objects created for them vs inodes that have an inode number allocated (for FUSE) but do not have an InodeBase object in memory. This will make it possible to unload Inode objects on demand to reduce memory usage. - Tracking of pending loads, and de-duplicating load requests. This ensures that only one Inode object ever exists for a given inode number / path. If a second request to load an Inode arrives when a previous load request is still in progress, InodeMap deals with this situation properly. - Better support for using Inode objects without FUSE. With the old code, attempts to interact with Inode objects without going through the FUSE dispatch (say, when processing a thrift call) could result in inconsistent state. New inodes created would not be put into the EdenDispatcher map, which could result in problems. - More convenient child inode access from TreeInode. With this change, the TreeInode class can easily tell which of its children are loaded. This makes it easier to do tasks which only need to operate on existing loaded inode state (for instance, dirstate computation). - Support for saving and loading state, to implement graceful restart.. InodeMap provides a central place to write out inode state on shutdown and restoring it on startup. Saved inodes can easily be restored to an "unloaded" state on startup. This code is not implemented yet as part of this diff, but it should be straightforward to add in future diffs. Reviewed By: bolinfest Differential Revision: D4318060 fbshipit-source-id: d9b16430fc8367e3516e788d1e991e5163aa6997
2016-12-23 02:34:54 +03:00
/**
* Internal API only for use by InodeMap.
*
* InodeMap will call this API when a child inode needs to be loaded.
* The TreeInode will call InodeMap::inodeLoadComplete() or
* InodeMap::inodeLoadFailed() when the load finishes.
*/
void loadChildInode(PathComponentPiece name, fuse_ino_t number);
/**
* Unload all unreferenced children under this tree (recursively).
*
* This walks the children underneath this tree, unloading any inodes that
* are unreferenced.
*/
void unloadChildrenNow();
private:
introduce a new InodeMap class Summary: This diff starts adding a new InodeMap class. This class will eventually consolidate the functionality of InodeNameMap plus the inode map stored in EdenDispatcher. This new class will bring several new benefits: - All inode mapping logic consolidated into a single class, with a single lock protecting the maps. - A well-defined model for loaded vs unloaded inodes. InodeMap explicitly tracks inodes that have InodeBase objects created for them vs inodes that have an inode number allocated (for FUSE) but do not have an InodeBase object in memory. This will make it possible to unload Inode objects on demand to reduce memory usage. - Tracking of pending loads, and de-duplicating load requests. This ensures that only one Inode object ever exists for a given inode number / path. If a second request to load an Inode arrives when a previous load request is still in progress, InodeMap deals with this situation properly. - Better support for using Inode objects without FUSE. With the old code, attempts to interact with Inode objects without going through the FUSE dispatch (say, when processing a thrift call) could result in inconsistent state. New inodes created would not be put into the EdenDispatcher map, which could result in problems. - More convenient child inode access from TreeInode. With this change, the TreeInode class can easily tell which of its children are loaded. This makes it easier to do tasks which only need to operate on existing loaded inode state (for instance, dirstate computation). - Support for saving and loading state, to implement graceful restart.. InodeMap provides a central place to write out inode state on shutdown and restoring it on startup. Saved inodes can easily be restored to an "unloaded" state on startup. This code is not implemented yet as part of this diff, but it should be straightforward to add in future diffs. Reviewed By: bolinfest Differential Revision: D4318060 fbshipit-source-id: d9b16430fc8367e3516e788d1e991e5163aa6997
2016-12-23 02:34:54 +03:00
void registerInodeLoadComplete(
folly::Future<InodePtr>& future,
PathComponentPiece name,
fuse_ino_t number);
folly::Future<InodePtr> startLoadingInodeNoThrow(
Entry* entry,
PathComponentPiece name,
fuse_ino_t number) noexcept;
folly::Future<InodePtr>
startLoadingInode(Entry* entry, PathComponentPiece name, fuse_ino_t number);
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
/** Translates a Tree object from our store into a Dir object
* used to track the directory in the inode */
static Dir buildDirFromTree(const Tree* tree);
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
/**
* Get a TreeInodePtr to ourself.
*
* This uses TreeInodePtr::newPtrFromExisting() internally.
*
* This should only be called in contexts where we know an external caller
* already has an existing reference to us. (Which is most places--a caller
* has to have a reference to us in order to call any of our APIs.)
*/
TreeInodePtr inodePtrFromThis() {
return TreeInodePtr::newPtrFromExisting(this);
}
folly::Future<folly::Unit>
rmdirImpl(PathComponent name, TreeInodePtr child, unsigned int attemptNum);
eden: merge overlay into the inode objects Summary: It was starting to get pretty complex to manage locking across the inodes, filedata, overlay and soon the journal, so as a simplifying step, this folds data that was tracked by the overlay into the TreeInode itself. This is the first diff in a short series for this. This one: 1. Breaks the persistent overlay information, so shutting down eden and bringing it back up will lose your changes (to be restored in the following diff) 2. Allows deferring materialization of file data in more cases 3. Allows renaming dirs. The approach here is now to keep just one source of information about the directory contents; when we construct a TreeInode we import this data from the Tree and then apply mutations to it locally. Each inode can be mutated indepdently from others; we only need to lock the 1, 2 or 3 participating inodes in the various mutation operations. I'll tackle persistence of the mutations in the following diff, but the high level plan for that (to help understand this diff) is to always keep the directory inodes for mutations alive as inode objects. We make use of the canForget functionality introduced by D3774269 to ensure that these don't get evicted early. On startup we'll load this information from the overlay area. This model simplifies some of the processing around reading dirs and looking up children. Since the overlay data now tracks the appropriate tree or content hash we can be much more lazy at materializing data, especially in the rename case. For example, renaming "fbcode" to "fbcod" doesn't require us to recursively materialize the "fbcode" tree. Depends on D3653706 Reviewed By: simpkins Differential Revision: D3657894 fbshipit-source-id: d4561639845ca93b93487dc84bf11ad795927b1f
2016-09-10 02:56:00 +03:00
folly::Synchronized<Dir> contents_;
/** Can be nullptr for the root inode only, otherwise will be non-null */
TreeInode::Entry* entry_{nullptr};
};
}
}