Commit Graph

135 Commits

Author SHA1 Message Date
Chad Austin
fc07c3b6e6 add an ObjectFetchContext interface
Summary:
Add a fetch context interface to ObjectStore that allows tracing cache
hits, backing store fetches, and fetch durations in the context of a
diff or checkout operation.

Reviewed By: simpkins

Differential Revision: D19135625

fbshipit-source-id: d0d8f134b1c89f7ba4971a404a46a69a1704ba5c
2020-02-05 13:15:01 -08:00
Chad Austin
a5cbdfeb1d remove the dead getChildByName function
Summary: #deadcode

Reviewed By: genevievehelsel

Differential Revision: D18822498

fbshipit-source-id: fd2dc63139758121c27a877e37c8e1aaf7f9c9b5
2019-12-20 16:14:18 -08:00
Joseph Friesen
921cc9d8a8 Merge TreeInode::create/mknod
Summary:
Refactor code to make Dispatch::create call TreeInode::mknod
and remove TreeInode::mknod, as it's redundant

Reviewed By: chadaustin

Differential Revision: D18002213

fbshipit-source-id: d0c73bbd6182226de54a49ef69b63b8fe7a8f9ee
2019-10-18 13:25:46 -07:00
Andres Suarez
fbdb46f5cb Tidy up license headers
Reviewed By: chadaustin

Differential Revision: D17872966

fbshipit-source-id: cd60a364a2146f0dadbeca693b1d4a5d7c97ff63
2019-10-11 05:28:23 -07:00
Genevieve Helsel
6a036808cc use callbacks in source control tree comparisons
Summary: Use callback to save ScmStatus instead of storing status inside a `TreeDiffer` object. Involes a bit of restructuring of some code to avoid circular dependencies and library creation. Mostly renames and file moves, some funtion moves as well.

Reviewed By: simpkins

Differential Revision: D17400466

fbshipit-source-id: fcd194a4c20204dffd3d11cd4a083564dc0ea938
2019-09-23 15:22:30 -07:00
Adam Simpkins
aa5e6c7295 update license headers in C++ files
Summary:
Update the copyright & license headers in C++ files to reflect the
relicensing to GPLv2

Reviewed By: wez

Differential Revision: D15487078

fbshipit-source-id: 19f24c933a64ecad0d3a692d0f8d2a38b4194b1d
2019-06-19 17:02:45 -07:00
Chad Austin
a9d9689d3d invalidate directory inodes properly on checkout
Summary:
After the kernel added readdir caching, my testing uncovered that Eden
was invalidating TreeInode entries incorrectly when new entries were
added. Change TreeInode to distinguish between directory entry changes
and removals (FUSE_NOTIFY_INVAL_ENTRY) and additions
(FUSE_NOTIFY_INVAL_INODE).

Reviewed By: strager

Differential Revision: D13870422

fbshipit-source-id: 2a6f25bfd9e77436a5aae639fedbfd8a445b2e05
2019-03-22 15:57:33 -07:00
Chad Austin
0c622944e3 eden: TreeInode::prefetch now also prefetches !materialized children
Summary:
This changes prefetch so that it loads all of the direct children of
the tree.  This improves `time ls -l bigdir` performance by 2x.

Reviewed By: wez

Differential Revision: D12888690

fbshipit-source-id: eb8c8274bd9c5b5edc94d7092a5feb492fad6d66
2019-01-08 16:56:24 -08:00
Chad Austin
5d98110a36 eden: move prefetch call from lookup to readdir
Summary:
We think that it shouldn't really be needed to perform
the prefetch call during lookup; for file inodes it doesn't buy
us much, and it should only really help for readdir.

This removes the prefetch call from lookup, instead prefetching
upon the first readdir() of a loaded TreeInode.

Reviewed By: simpkins

Differential Revision: D12896022

fbshipit-source-id: 0209eb64bd522daf5f7461dffccd1312d32a1554
2019-01-08 16:56:24 -08:00
Chad Austin
5222e339e6 remove EdenFileHandle and FileInode::open
Summary: Title says it all.

Reviewed By: strager

Differential Revision: D13325746

fbshipit-source-id: 22f1b12ba0bf47eba62c2312e5069c45b1c28ef3
2018-12-12 17:10:29 -08:00
Chad Austin
9dbdccb8ed remove TreeInodeDirHandle, fix unmount while dir handle is open
Summary:
Stop holding a reference count to the TreeInode while a directory
handle is open. This allows eden to shut down while a directory handle
is open.

Reviewed By: strager

Differential Revision: D13287701

fbshipit-source-id: a24f32a1ac40b6c19bc5864aa5f5785f3016361b
2018-12-04 16:37:41 -08:00
Chad Austin
c55edc9036 route readdir straight to TreeInode
Summary:
Send readdir requests to TreeInode. This may not sound like a good
idea: the FUSE documentation suggests that stateful directory handles
are required to implement correct readdir semantics under concurrent
deletes and renames. However, the 63-bit offset value is treated as a
cookie that is passed from one readdir call into the next, and 63 bits
should be sufficient to implement readdir concurrent with
rename/unlink. So move readdir's implementation into TreeInode in
preparation for the complete removal of TreeInodeDirHandle.

Reviewed By: strager

Differential Revision: D13287664

fbshipit-source-id: c0d615675edd9b83353534468a69b89068bba923
2018-12-04 16:37:41 -08:00
Chad Austin
5029338c62 remove overlay timestamp migration logic
Summary:
Now that the Overlay no longer serializes timestamps, remove all of
the special-case migration logic.

Reviewed By: strager

Differential Revision: D13144764

fbshipit-source-id: 713a4bfcde9003a8d5a28837cb530b05a9017c22
2018-11-28 15:44:58 -08:00
Chad Austin
308ef1ddb8 stop reading timestamps from the overlay
Summary:
Eden has used the InodeMetadataTable as the authoritative source for
timestamp metadata for over six months. I think we can safely assume
that anyone at this point who has old inodes in the overlay that don't
have corresponding entries in the inode metadata table are fine with
those timestamps being reset to the last checkout time.

Reviewed By: strager

Differential Revision: D13144735

fbshipit-source-id: 06a9a8835ea83c98fb6a165e4c8d5c3c6b28ad84
2018-11-26 11:59:25 -08:00
Chad Austin
3091df6e68 stop writing timestamps into the overlay
Summary:
Eden has used the InodeMetadataTable as the primary source of
timestamp data for more than six months. Stop writing timestamps into
the overlay, since they will never be used.

Reviewed By: strager

Differential Revision: D13144696

fbshipit-source-id: e36423036228e89dd2a986e6bacfa74553c17a92
2018-11-26 11:59:25 -08:00
Chad Austin
74ed419fd4 remove the dead deep recursion option to loadMaterializedChildren
Summary:
Eden used to load materialized entries at startup. It no longer does
and this code is dead.

Reviewed By: wez

Differential Revision: D12896210

fbshipit-source-id: 398363724a661f87208cf05313e61755a451edb7
2018-11-01 18:23:39 -07:00
Chad Austin
2a6dd2879d folly::Optional -> std::optional
Summary: Eden's on C++17 so fully cross the rubicon!

Reviewed By: strager

Differential Revision: D10498200

fbshipit-source-id: 4e2af5a8d5cef9a106e8c05e6f93ea9e5b8e696e
2018-10-23 18:51:59 -07:00
Chad Austin
be065e5997 allow reading xattrs from files after getxattr is called on a directory
Summary:
Eden supports reading the SHA-1 of a file via getxattr. Unfortunately,
it returned ENOSYS if you called getxattr on a directory inode. This
caused FUSE to fail all future getxattr requests with EOPNOTSUPP.

In addition to fixing that, this diff makes our xattr handling a
little more consistent across inodes:

- setxattr() always fails with ENOSYS
- removexattr() always fails with ENOSYS
- listxattr() is always handled by the corresponding inode class
- getxattr() is always handled by the corresponding inode class

Differential Revision: D10437723

fbshipit-source-id: a1ea1e92d3412abb15e91057becea3160a17f1e2
2018-10-22 20:27:26 -07:00
Chad Austin
b38171cb08 move fsync and flush from EdenFileHandle to FileInode
Summary: Clip more logic from EdenFileHandle.

Reviewed By: wez

Differential Revision: D10187239

fbshipit-source-id: f11090e23bd1d6e61414e4d9455509e0dca305f2
2018-10-08 11:13:38 -07:00
Chad Austin
5567363018 add a function that only unloads children unreferenced by FUSE
Summary: Add yet another unloading code path... This one is used for fast checkouts.

Reviewed By: strager

Differential Revision: D9781956

fbshipit-source-id: ae89377aea823f94e2ec1bcc2fa209c8f9bc821c
2018-09-12 14:37:25 -07:00
Chad Austin
5070d76451 fix TreeInode::unloadChildrenNow so it unloads trees
Summary:
unloadChildrenNow would only unload files or trees at the leaf. Apply
the fixes from D8302998 to unloadChildrenNow, which is only called
during takeover.

Reviewed By: strager

Differential Revision: D9774970

fbshipit-source-id: c2f4d1e6b838cc3b9e99eb8786114e643128a519
2018-09-12 14:37:24 -07:00
Chad Austin
262aff7bda Have eden debug unload unload trees too
Summary:
Rewrite the unload code to avoid UB (comparing freed InodeBase
pointers) and unload tree inodes too.

Reviewed By: simpkins

Differential Revision: D8302998

fbshipit-source-id: c3c3b1a65209735056fbc66216e4f01eafd65492
2018-08-31 11:22:30 -07:00
Chad Austin
f87e6bcb93 add a safe getMetadata method to InodeBase
Summary:
Prior to this diff, to read the inode metadata from an InodeBase, you
had to cast it to a FileInode or TreeInode. Add a public getMetadata
call that acquires the appropriate lock depending on whether it's a
FileInode or TreeInode.

Reviewed By: simpkins

Differential Revision: D9327180

fbshipit-source-id: 66594fb639f1ea72a0e48dcb79234e14d87ec4a2
2018-08-23 16:55:58 -07:00
Chad Austin
de9416e41c stop writing to the overlay header on unload
Summary:
Now that timestamps are read from the inode metadata table, and users
aren't likely to run a pre-metadata-table version, the timestamp data
in the overlay header's no longer needs to be written. So remove that
code which has the bonus of making unloading faster.

Reviewed By: wez

Differential Revision: D9318044

fbshipit-source-id: 27a9a9ee954003940209819466932237a81f8929
2018-08-16 11:52:29 -07:00
Adam Simpkins
ee88d00389 rename updateOverlayHeader() to updateOverlayHeaderIfNeeded()
Summary:
The `updateOverlayHeader()` only updates the overlay data if the inode is
materialized.  This updates the name to clarify that.

(This function name change was previously part of D8884795, and I'm just
splitting it into its own separate diff.)

Reviewed By: bolinfest

Differential Revision: D9011358

fbshipit-source-id: 6024d64a1dee0b5d741bec32ed88f6c8f8dd8a9a
2018-07-27 13:22:58 -07:00
Chad Austin
8e556e4979 Don't derive TreeInodeState from DirContents
Summary:
Per a conversation with simpkins when code reviewing D7882648, this
diff removes the inheritance relationship between TreeInodeState and
DirContents. It doesn't change the binary layout of anything, but
defines DirContents as a typedef of PathMap<DirEntry>.

Reviewed By: strager

Differential Revision: D8232052

fbshipit-source-id: a2166f3ca2ab90fabbded0e48307b8a92a2b0250
2018-06-13 11:53:33 -07:00
Chad Austin
feb2da11d6 update mtime on directories when renaming
Summary:
This fixes a bug simpkins pointed out in D6891479 - we weren't
updating mtime and ctime on renames.

Reviewed By: simpkins

Differential Revision: D7937303

fbshipit-source-id: 08fd8f4fe5d99d33e9f7629965d6146330c8f35b
2018-06-01 11:39:01 -07:00
Chad Austin
043aee80a0 ensure that inodes access and update metadata while their corresponding locks are held
Summary:
Per code review feedback from D6891479, this diff enforces that
metadata writes and reads are done while the corresponding inode's
state lock is held.

Reviewed By: simpkins

Differential Revision: D7884463

fbshipit-source-id: d0e7a95415c280441276452ece7233d4cbf5e942
2018-06-01 11:39:01 -07:00
Chad Austin
e939a6a821 Split Dir into its own type and move TreeInodeState out of TreeInode
Summary:
Like D7867399, split TreeInode's synchronized state into a top-level
class. This is a step towards using the type system to perform
lock-safe metadata updates.

Reviewed By: simpkins

Differential Revision: D7882648

fbshipit-source-id: 27262df8ed9137c8478c68ebf4c4f13878655754
2018-06-01 09:38:45 -07:00
Chad Austin
84b577b5f1 fit Dir Entry in five words
Summary:
Eden will often have a significant number of trees loaded - this saves
8 bytes per entry per loaded TreeInode. It also makes it clear that
once an inode pointer is assigned, the inode number is redundant.

Reviewed By: simpkins

Differential Revision: D7869662

fbshipit-source-id: 21a8266ff5189d3ba9cb614a325cc9d8c3ca305e
2018-06-01 09:38:45 -07:00
Chad Austin
8bda0847d3 prep for shrinking Entry to 5 words
Summary:
Shrinking Entry to five words was a bit tricky with some subtle
situations, so I split it into two diffs to verify the tests passed
after each stage.

This diff replaces folly::Optional<Hash> with a boolean and Hash.

Reviewed By: simpkins

Differential Revision: D7869502

fbshipit-source-id: 2df109472d9565e96e8621407f62a63b4f1dbcad
2018-06-01 09:38:45 -07:00
Chad Austin
5409f230eb have Overlay track nextInodeNumber_ instead of InodeMap
Summary:
The Overlay is the natural home for nextInodeNumber_ now that every
directory allocates inode numbers for its children right away. This
also simplifies serializing nextInodeNumber_ to disk in the following
diff.

Reviewed By: simpkins

Differential Revision: D8192442

fbshipit-source-id: 9b776a73c8d7653002b55985d592b1746e52f878
2018-05-31 01:46:15 -07:00
Chad Austin
df4dda2dfe allow changing uid/gid on files and directories
Summary:
Relax the restriction on changing uid/gid on inodes.  We'll see what
cans of worms this opens I guess.  (Landing this is low priority, but
might be important for making some of the existing tooling in fbsource
and www work.)

Reviewed By: simpkins

Differential Revision: D7768655

fbshipit-source-id: 95fe02fe7ddc001335dbdb34e16a989a85820240
2018-05-24 00:15:54 -07:00
Chad Austin
d4e186f995 store FileInode and TreeInode timestamps in the InodeTable
Summary: Store tree and file timestamps in the InodeTable so they persist across runs.

Reviewed By: simpkins

Differential Revision: D6891479

fbshipit-source-id: 1c9e6266375aceeaf293a81e73cf7f5334dbc32d
2018-05-22 11:22:19 -07:00
Chad Austin
c22dd7924d mark a bunch of folly::Future<folly::Unit> functions with FOLLY_NODISCARD
Summary:
To avoid bugs similar to the ones fixed in D7781691, mark a bunch of
folly::Future<folly::Unit> functions with FOLLY_NODISCARD.

Reviewed By: simpkins

Differential Revision: D7782224

fbshipit-source-id: 23ba42aa63011cc33e5a6e18d5bc6d00403a78d3
2018-04-26 21:00:25 -07:00
Chad Austin
45c33d5cfd Lock down access to the .eden directory after mount startup
Summary:
Disallow any kind of mutation operation inside of the .eden directory.  We had some
code in place to prevent some of this already, but errors (including EPERM) weren't
passed out from unlink and rename out to FUSE.

Reviewed By: simpkins

Differential Revision: D7781691

fbshipit-source-id: aaecf13779eca75d6ee8765fc8bb3727ce9341de
2018-04-26 21:00:25 -07:00
Chad Austin
a9d4c071ef give InodeBase a mode_t instead of dtype_t
Summary: The InodeTable work will homogenize mode_t storage, access, and modification across file and tree inodes.  In preparation, have InodeBase keep track of the initial mode bits instead of a dtype_t.

Reviewed By: simpkins

Differential Revision: D7031924

fbshipit-source-id: f2e6e4467cecfc0ca06ad998cce0af18a99cc251
2018-04-26 16:20:05 -07:00
Chad Austin
13dc5d1d51 minor prefactoring split out from later diffs
Summary: Also print the inode path if the assertions in EXPECT_FILE_INODE fail.

Reviewed By: simpkins

Differential Revision: D7035517

fbshipit-source-id: 6c50acb588d1c985c7e7a1586c06f04a657698f9
2018-04-24 18:53:19 -07:00
Chad Austin
42e2ffad76 immediately save inode numbers when allocating a TreeInode
Summary:
When we start storing permission changes and other inode
metadata keyed on inode number, we will need to remember inode numbers
before they're accessed to handle the case where the Eden process dies
before the inode is unloaded.  Otherwise, data could be left in an
inconsistent state and some writes could be missed.

This change depends on saving (and forgetting) directories being
cheap.

Reviewed By: simpkins

Differential Revision: D7598094

fbshipit-source-id: 9ab127b30a9c09ab62aa08b50c13b3eaa40be60d
2018-04-20 18:52:17 -07:00
Chad Austin
42808d2ced centralize overlay dir access
Summary:
I'm about to change the code paths through which directories
are saved to and loaded from persistent storage.  To help, consolidate
those operations.

Reviewed By: simpkins

Differential Revision: D7597978

fbshipit-source-id: 9cd20027746b0a372a97b7f5daf07c1b54e6b9ce
2018-04-13 10:58:39 -07:00
Adam Simpkins
7c2d0b13b1 store the Overlay using unique_ptr rather than shared_ptr
Summary: Each Overlay object should be owned only by its EdenMount.

Reviewed By: chadaustin

Differential Revision: D7543294

fbshipit-source-id: 6db40fea31ce298c61d047cba9165887e32926a1
2018-04-09 12:21:03 -07:00
Adam Simpkins
f334d7ab79 reduce code duplication in TreeInode create methods
Summary:
Most of the logic in TreeInode::create(), symlink() and mknod() was very
similar.  This adds a TreeInode::createImpl() helper method that these three
functions can share.

This also updates TreeInode::create() to throw EEXIST if the file in question
does exist, rather than using EDEN_BUG().  I believe it is possible (but
unlikely) for this to occur in practice since inode invalidation operations
triggered by a checkout are now processed asynchronously.

Reviewed By: chadaustin

Differential Revision: D7411500

fbshipit-source-id: 85d97995139eee6bff96381561fc28e76d7a2b7c
2018-03-28 13:13:52 -07:00
Chad Austin
e56b25df9f remove loading bit from TreeInode::Entry
Summary: After the changes in D7052470 and D7346263, this bit is no longer necessary.

Reviewed By: simpkins

Differential Revision: D7401540

fbshipit-source-id: 04f3ae0376625d4e9584f31932f46774eb5caba4
2018-03-27 13:23:13 -07:00
Adam Simpkins
b324323ad4 make inode unloading methods non-const
Summary:
Update InodeMap::onInodeUnreferenced() to take a pointer to a non-const
InodeBase.  This allows the methods it calls, including
InodeBase::updateOverlayHeader() to be non-const.

Using non-const InodeBase objects seems to make sense here since the inode is
in the process of being destroyed.

In a future diff I plan to update FileInode::updateOverlayHeader() to share the
same code as normal file methods to ensure that the overlay file is open.  This
modifies the FileInode state's open refcount, so it is useful to have this
method be non-const for that purpose.

Reviewed By: chadaustin

Differential Revision: D7407424

fbshipit-source-id: 541656c7b9b283c5e5650445de5bbdbaae3fc57f
2018-03-27 11:23:42 -07:00
Chad Austin
c80f10ad03 decouple whether an entry has an inode number from materialization status
Summary:
Decouple inode number assignment from materialization status.
The idea is that we will always assign entries an inode number and
track whether an entry is materialized otherwise.  This is necessary
to give consistent inode values across remounts.

Reviewed By: simpkins

Differential Revision: D7052470

fbshipit-source-id: 80d3f2a2938463198a3132182537e6223c79d509
2018-03-23 18:21:21 -07:00
Chad Austin
bf31028299 test and fix for takeover bug when trees are referenced across a takeover
Summary:
Verify Eden handles looking up an inode by number after graceful
restart and checkout.

Reviewed By: simpkins

Differential Revision: D7346263

fbshipit-source-id: 876b4837708da9ac31f72c06e7defc797fe126f3
2018-03-23 15:06:22 -07:00
Chad Austin
c176bf06fd simplify startLoadingInode
Summary: Small refactoring prior to a fix in a later diff.

Reviewed By: simpkins

Differential Revision: D7358218

fbshipit-source-id: 198160df1aa0dc65b917c5a96c2998c3675b03fe
2018-03-23 15:06:22 -07:00
Adam Simpkins
bfcf4c574a remove the fusell namespace
Summary: Move everything in the `facebook::eden::fusell` namespace to `facebook::eden`

Reviewed By: chadaustin

Differential Revision: D7314458

fbshipit-source-id: db56d3e5fb898235e1376ac76077cf780d9b4698
2018-03-19 17:01:52 -07:00
Adam Simpkins
da3c070d70 rename FileHandle to EdenFileHandle
Summary:
This helps distinguish `facebook::eden::FileHandle` from
`facebook::eden::fusell::FileHandle`, and will make it possible to eliminate
the separate `facebook::eden::fusell` namespace in the future.

Reviewed By: chadaustin

Differential Revision: D7314456

fbshipit-source-id: 0869427766fe666d119a59c7df1c97825a8ad36e
2018-03-19 12:55:17 -07:00
Chad Austin
82d2a71c41 Add a loading bit to TreeInode::Entry
Summary:
There's some code that uses Entry::hasInodeNumber to decide whether an
inode is in the process of being loaded.  This change is a
prerequisite for D7052470.

Reviewed By: simpkins

Differential Revision: D7311182

fbshipit-source-id: e04a89c814950e6ce33e73b963b3ffb0f0e353b8
2018-03-19 11:22:00 -07:00