Commit Graph

120 Commits

Author SHA1 Message Date
Sergey Zhupanov
0d6c3a31cd Changed TreeInode::loadGitIgnoreThenDiff() to properly handle symlinks to gitignore.
Summary: Added TreeInode::loadGitIgnoreThenDiff() symlink handling with tests.

Reviewed By: simpkins

Differential Revision: D6659654

fbshipit-source-id: 293c913ea56b5b770a051efd78e1e57497c360bd
2018-01-05 15:51:04 -08:00
Wez Furlong
6ff492d11c remove dep on libfuse
Summary:
This serves a few purposes:

1. We can avoid some conditional code inside eden if we know that
   we have a specific fuse_kernel.h header implementation.
2. We don't have to figure out a way to propagate the kernel
   capabilities through the graceful restart process.
3. libfuse3 removed the channel/session hooks that we've been
   using thus far to interject ourselves for mounting and
   graceful restarting, so we were already effectively the
   walking dead here.
4. We're now able to take advtange of the latest aspects of
   the fuse kernel interface without being tied to the implementation
   of libfuse2 or libfuse3.  We're interested in the readdirplus
   functionality and will look at enabling that in a future diff.

This may make some things slightly harder for the more immediate
macOS port but I belive that we're in a much better place overall.

This diff is relatively mechanical and sadly is (unavoidably) large.

The main aspects of this diff are:

1. The `fuse_ino_t` type was provided by libfuse so we needed to
   replace it with our own definition.  This has decent penetration
   throughout the codebase.
2. The confusing `fuse_file_info` type that was multi-purpose and
   had fields that were sometimes *in* parameters and sometimes *out*
   parameters has been removed and replaced with a simpler *flags*
   parameter that corresponds to the `open(2)` flags parameter.
   The *out* portions are subsumed by existing file handle metadata
   methods.
3. The fuse parameters returned from variations of the `LOOKUP` opcode
   now return the fuse kernel type for this directly.  I suspect
   that we may need to introduce a compatibility type when we revisit
   the macOS port, but this at least makes this diff slightly simpler.
   You'll notice that some field and symbol name prefixes vary as
   a result of this.
4. Similarly for `setattr`, libfuse separated the kernel data into
   two parameters that were a little awkward to use; we're now just
   passing the kernel data through and this, IMO, makes the interface
   slightly more understandable.
5. The bulk of the code from `Dispatcher.cpp` that shimmed the
   libfuse callbacks into the C++ virtual methods has been removed
   and replaced by a `switch` statement based dispatcher in
   `FuseChannel`.   I'm not married to this being `switch` based
   and may revise this to be driven by an `unordered_map` of
   opcode -> dispatcher method defined in `FuseChannel`.  Regardless,
   `Dispatcher.cpp` is now much slimmer and should be easier to
   replace by rolling it together into `EdenDispatcher.cpp` in
   the future should we desire to do so.
6. This diff disables dispatching `poll` and `ioctl` calls.  We
   didn't make use of them and their interfaces are a bit fiddly.
7. `INTERRUPT` is also disabled here.  I will re-enable it in
   a follow-up diff where I can also revise how we track outstanding
   requests for graceful shutdown.
8. I've imported `fuse_kernel.h` from libfuse.  This is included
   under the permissive 2-clause BSD license that it allows for
   exactly this integration purpose.

Reviewed By: simpkins

Differential Revision: D6576472

fbshipit-source-id: 7cb088af5e06fe27bf22a1bed295c18c17d8006c
2018-01-02 16:36:16 -08:00
Adam Simpkins
e9567ba6d4 improve short-circuiting logic on dry-run checkouts
Summary:
Update the logic for dry-run checkouts to exit early whenever we hit a
directory that is unmodified from the original source control state.  In this
case we can skip this entire subdirectory since we know there cannot be any
conflicts.

This should make dry-run checkout operations much faster in cases where there
are lots of changes between the source and destination source control trees,
but the working directory state has relatively few modified files.

Reviewed By: bolinfest

Differential Revision: D6605282

fbshipit-source-id: b5423749f3d47b10ed8d599ffaa0667c72fbaec2
2017-12-19 20:54:32 -08:00
Sergey Zhupanov
58b472b9d0 Added type identification capability to InodeBase.
Summary: Added type identification capability to InodeBase and its descendants FileInode and TreeInode.

Reviewed By: simpkins

Differential Revision: D6564902

fbshipit-source-id: ce9300102d6d6d1c42616eb1e32042f21f6e6cce
2017-12-14 16:41:39 -08:00
Chad Austin
2f22a9f534 introduce a type representing the unbounded eden cpu pool
Summary:
Add EdenCPUThreadPool and UnboundedQueueThreadPool types to make it clearer
that it's always okay for prefetch, deferred diff entry, and hg import to
shuttle work back to the main thread pool.

This diff changes no behavior - it just makes some invariants explicit.

Reviewed By: wez, simpkins

Differential Revision: D6504117

fbshipit-source-id: 3400ad55c00b3719ecba31807fd992442f622cd9
2017-12-12 12:35:35 -08:00
Sergey Zhupanov
b6394ac357 Added user and general system level gitignore
Summary:
Added to Eden capability to incorporate default user and general system level gitignore files.
NOTE: Work in progress, sending the review out to calibrate/ensure I am on right track.

Reviewed By: simpkins

Differential Revision: D6482863

fbshipit-source-id: 9834ca1a577a9599a1f8cb2243dca4e714866be8
2017-12-08 12:52:51 -08:00
Chad Austin
d112d63870 remove all direct calls to clock_gettime and system_clock::now
Summary:
Follow-up to comments in D6466209.  All access to the clock goes
through the Clock interface, making time deterministic in unit tests.

Reviewed By: simpkins

Differential Revision: D6477973

fbshipit-source-id: 24e51bdb52d0d079b34d91598d2e787d361f2525
2017-12-05 10:06:50 -08:00
Chad Austin
70f1bba3d6 test that asserts a new file inode's timestamps match the creation time, not the last checkout time
Summary: Follow-up from D6366189. First use of the new FakeClock!

Reviewed By: simpkins

Differential Revision: D6466209

fbshipit-source-id: 4d4d8a9a83df2bee11149e7a0cbddaaf734d0e04
2017-12-05 10:06:50 -08:00
Adam Simpkins
f3d9dcbc4d speed up hg update --clean operations
Summary:
Update TreeInode::computeCheckoutActions() to short circuit on non-materialized
directories that are identical to both the destination and source state.  This
speeds up `hg update --clean` operations by allowing us to skip portions of the
tree that do not require changes.

This does not have much effect for update operations without `--clean`: those
already short-circuited the operation in `TreeInode::processCheckoutEntry()`
when the source and destination hashes were the same.  That code cannot
short-circuit in the forced update case, since the forced update may have to
revert local modifications.

Reviewed By: bolinfest

Differential Revision: D6455970

fbshipit-source-id: 393acb3272745751a56e06dba0c7505ff2bfad44
2017-11-30 20:35:32 -08:00
Michael Bolin
5e2afa735f Change how the UNTRACKED_ADDED conflict and merges are handled.
Summary:
Previously, we used the Mercurial code `g` when faced with an `UNTRACKED_ADDED`
file conflict, but that was allowing merges to silently succeed that should not
have. This revision changes our logic to use the code `m` for merge, which
unearthed that we were not honoring the user's `update.check` setting properly.

Because we use `update.check=noconflict` internally at Facebook, we changed the
Eden integration tests to default to verifying Hg running with this setting. To
support it properly, we had to port this code from `update.py` in Mercurial to
our own `_determine_actions_for_conflicts()` function:

```
if updatecheck == 'noconflict':
    for f, (m, args, msg) in actionbyfile.iteritems():
        if m not in ('g', 'k', 'e', 'r', 'pr'):
            msg = _("conflicting changes")
            hint = _("commit or update --clean to discard changes")
            raise error.Abort(msg, hint=hint)
```

However, this introduced an interesting issue where the `checkOutRevision()`
Thrift call from Hg would update the `SNAPSHOT` file on the server, but
`.hg/dirstate` would not get updated with the new parents until the update
completed on the client. With the new call to `raise error.Abort` on the client,
we could get in a state where the `SNAPSHOT` file had the hash of the commit
assuming the update succeeded, but `.hg/dirstate` reflected the reality where it
failed.

To that end, we changed `checkOutRevision()` to take a new parameter,
`checkoutMode`, which can take on one of three values: `NORMAL`, `DRY_RUN`, and
`FORCE`. Now if the user tries to do an ordinary `hg update` with
`update.check=noconflict`, we first do a `DRY_RUN` and examine the potential
conflicts. Only if the conflicts should not block the update do we proceed with
a call to `checkOutRevision()` in `NORMAL` mode.

To make this work, we had to make a number of changes to `CheckoutAction`,
`CheckoutContext`, `EdenMount`, and `TreeInode` to keep track of the
`checkoutMode` and ensure that no changes are made to the working copy when a
`DRY_RUN` is in effect.

One minor issue (for which there is a `TODO`) is that a `DRY_RUN` will not
report any `DIRECTORY_NOT_EMPTY` conflicts that may exist. As `TreeInode` is
implemented today, it is a bit messy to report this type of conflict without
modifying the working copy along the way.

Finally, any `UNTRACKED_ADDED` conflict should cause an update to
abort to match the behavior in stock Mercurial if the user has the following
config setting:

```
[commands]
update.check = noconflict
```

Though the original name for this setting was:

```
[experimental]
updatecheck = noconflict
```

Although I am on Mercurial 4.4.1, the `update.check` setting does not seem to
take effect when I run the integration tests, but the `updatecheck` setting
does, so for now, I set both in `hg_extension_test_base.py` with a `TODO` to
remove `updatecheck` once I can get `update.check` to do its job.

Reviewed By: simpkins

Differential Revision: D6366007

fbshipit-source-id: bb3ecb1270e77d59d7d9e7baa36ada61971bbc49
2017-11-29 21:50:34 -08:00
Chad Austin
200fefb024 set in-memory ctime properly when creating new FileInodes
Summary:
This corrects a bug where the in-memory timestamp for new files would
be set to the last checkout time instead of when the file was created.

Reviewed By: simpkins

Differential Revision: D6366189

fbshipit-source-id: c5fa8c779726d3a75c2d57b2a161293297eb9271
2017-11-21 09:21:29 -08:00
Chad Austin
7389babe42 update comments regarding O_EXCL behavior
Summary:
Per a discussion with wez, let's make it explicit in the comments that the kernel handles
O_EXCL for us.

Reviewed By: simpkins

Differential Revision: D6365157

fbshipit-source-id: 638f67cd89a9450ff084e0ef77c731ef738bf518
2017-11-21 09:21:29 -08:00
Chad Austin
9217c5b56b refactor inode creation a bit, remove finishCreate()
Summary:
This is preparation for a change I'm preparing that makes the state
transitions in FileInode clearer and safer.

Reviewed By: wez

Differential Revision: D6304272

fbshipit-source-id: 493ee80517443432f790abf9806000eecb03651c
2017-11-16 00:29:24 -08:00
Chad Austin
587d0ad1ee Move Journal locking inside of Journal itself
Summary:
This is follow-up to the lock ordering issues in
StreamingSubscriber.  The Journal locks are now finer-grained and no
locks are held while the subscribers are invoked.  This change
prevents future deadlocks.

Reviewed By: wez

Differential Revision: D6281410

fbshipit-source-id: 797c164395831752f61cc15928b6d4ce4dab1b68
2017-11-10 19:57:49 -08:00
Adam Simpkins
107c2e1753 add debug logging for TreeInode::diff()
Summary:
Add log messages that will help debug the behavior of `TreeInode::diff()`

I added this since I saw a case where `getScmStatus()` was returning a file as
modified when it did not appear to have mode or contents changes.
Unfortunately we did not have any existing debug log messages for this, and the
problem went away after restarting edenfs, so I wasn't able to track it down.
Having these log messages will help debug this in the future if we run into it
again.

Reviewed By: bolinfest

Differential Revision: D6281905

fbshipit-source-id: 736028d6efe46a387bed6bf98d971685f83da3ec
2017-11-08 21:39:07 -08:00
Chad Austin
536bd6e298 Change FileInode::readAll's public API to be async
Summary: To fix up the invariants in FileInode's API, we intend to remove ensureDataLoaded() and materializeForWrite().  But first I'm going to see if there is any pain caused by removing their calls.

Reviewed By: simpkins

Differential Revision: D6234049

fbshipit-source-id: c39c6d018d164b32903414d3750b875a897af210
2017-11-03 17:51:42 -07:00
Chad Austin
8b9261f2a1 run clang-format across all C++ files
Summary:
Per discussion with bolinfest, this brings Eden in line with clang-format.

This diff was generated with `find . \( -iname '*.cpp' -o -iname '*.h' \) -exec bash -c "yes | arc lint {}" \;`

Reviewed By: bolinfest

Differential Revision: D6232695

fbshipit-source-id: d54942bf1c69b5b0dcd4df629f1f2d5538c9e28c
2017-11-03 16:02:03 -07:00
Chad Austin
f79672e8ec clean up partially-written symlinks if symlink creation fails
Summary: If writing out the symlink file in the overlay fails, clean up the file before releasing the lock.

Reviewed By: wez

Differential Revision: D6234214

fbshipit-source-id: 91ccf917a26cc1a73c3963dea7dd87364857fa03
2017-11-03 15:25:34 -07:00
Adam Simpkins
07b61591ca avoid holding TreeInode locks while locking the Journal
Summary:
Most functions in TreeInode that call Journal::addDelta() release the
TreeInode::contents_ lock before locking the journal.  However, `doRename()`
locked the journal while still holding the contents_ locks for the directories
affected by the rename.

This updates `doRename()` to make sure we release the TreeInode locks properly
before attempting to update the journal.  This should help ensure that we don't
have any lock ordering issues here.

Reviewed By: bolinfest

Differential Revision: D6212075

fbshipit-source-id: 84503acf47e9d97b81075932326825832cef9514
2017-11-01 20:43:43 -07:00
Adam Simpkins
2e6ed25612 flush kernel caches properly for empty directories removed by checkout
Summary:
When performing an source control checkout operation, we attempt to remove
directories that are empty after the checkout.  However, this code path was
missing a call to flush the kernel cache for these directories.

As a result, even though eden thought the directory not longer existed, and
would not report it in `readdir()` results, the kernel would return stale
information from its cache when explicitly accessing this path.

Reviewed By: bolinfest

Differential Revision: D6151543

fbshipit-source-id: 6031feb268ff6f980c885efb26c3d43243dec3f4
2017-10-25 16:51:56 -07:00
Adam Simpkins
35c50ea4e3 add more debug logging of checkout operations
Summary:
Add a few extra debug logs to record `processCheckoutEntry()` and
`saveOverlayPostCheckout()` calls.

Reviewed By: bolinfest

Differential Revision: D6151544

fbshipit-source-id: ca6faa8fd1fe53df1e70305f5527360c918841d1
2017-10-25 16:51:56 -07:00
Wez Furlong
aa7a07f209 remove warning about already loaded inodes
Summary:
Since adding prefetch support, we're triggering this code
each time we perform a lookup operation on a dir which means that
the eden logs are made a bit more noisy with this output than
previously.

So, let's remove the warning.

Reviewed By: simpkins

Differential Revision: D5905454

fbshipit-source-id: 0abc9b42284a5224cef908293ab377d8858977ec
2017-10-17 13:07:50 -07:00
Wez Furlong
364398cf94 ObjectStore now returns shared_ptr<const>
Summary:
Originally I thought this would help move towards removing a
`future.get()` call from FileInode, but it turned out to not make a difference
to that code.

It does make it a bit less of a chore to deal with the Journal related diff
callbacks added in D5896494 though, and is a move towards a future where we
could potentially return cached and shared instances of these objects.

This diff is a mechanical change to alter the return type so that we can share
instances returned from the object store interface.  It doesn't change any
functionality.

Reviewed By: simpkins

Differential Revision: D5919268

fbshipit-source-id: efe4b3af74e80cf1df20e81b4386450b72fa2c94
2017-09-29 16:54:05 -07:00
Chad Austin
3fb5680152 Rename ConflictType::MODIFIED to ConflictType::MODIFIED_MODIFIED
Summary: Per wez, this makes the MODIFIED case consistent with the other conflict types (e.g. local_remote).  Side benefit of avoiding some naming conflicts in the Haskell/Rust thrift tooling.

Reviewed By: wez

Differential Revision: D5882327

fbshipit-source-id: 3ec68c44d8c8a5c5675f1ced3842d29376d46fe2
2017-09-21 16:54:37 -07:00
Chad Austin
24c0fe9e8f prefetch inode contents when they're looked up
Reviewed By: wez

Differential Revision: D5817915

fbshipit-source-id: 3820f635cc6251ae5e13a4e214ba66df25ab6687
2017-09-19 11:10:11 -07:00
Wez Furlong
3da68e5adc dumb merge of MountPoint into EdenMount
Summary:
This is a mechanical and dumb move of the code from MountPoint
and into the EdenMount class.

Of note, it doesn't merge together the two different state/status fields
into a unified thing; that will be tackled in a follow on diff.

Reviewed By: bolinfest

Differential Revision: D5778212

fbshipit-source-id: 6e91a90a5cc760429d87a475ec12f81b93f87be0
2017-09-08 19:25:34 -07:00
Wez Furlong
467c417ccb re-organize the fuse Channel and Session code
Summary:
The higher level goal is to make it easier to deal
with the graceful restart scenario.

This diff removes the SessionDeleter class and effectively renames
the Channel class to FuseChannel.  The FuseChannel represents
the channel to the kernel; it can be constructed from a fuse
device descriptor that has been obtained either from the privhelper
at mount time, or from the graceful restart procedure.  Importantly
for graceful restart, it is possible to move the fuse device
descriptor out of the FuseChannel so that it can be transferred
to a new eden process.

The graceful restart procedure requires a bit more control over
the lifecycle of the fuse event loop so this diff also takes over
managing the thread pool for the worker threads.  The threads
are owned by the MountPoint class which continues to be responsible
for starting and stopping the fuse session and notifying EdenServer
when it has finished.  A nice side effect of this change is that
we can remove a couple of inelegant aspects of the integration;
the stack size env var stuff and the redundant extra thread
to wait for the loop to finish.

I opted to expose the dispatcher ops struct via an `extern` to
simplify the code in the MountPoint class and avoid adding special
interfaces for passing the ops around; they're constant anyway
so this doesn't feel especially egregious.

Reviewed By: bolinfest

Differential Revision: D5751521

fbshipit-source-id: 5ba4fff48f3efb31a809adfc7787555711f649c9
2017-09-08 19:25:34 -07:00
Jyothsna Konisa
72b61a5ddc Changes to return unloaded inode count for TreeInode::unloadChildrenNow
Summary:
1.Modified `TreeInode::unloadChildrenNow()` to return number of inodes that have been unloaded.
2.Modified `EdenServiceHandler::unloadInodeForPath()` to return number of inodes that are unloaded.

Reviewed By: simpkins

Differential Revision: D5627539

fbshipit-source-id: 4cdb0433dced6bf101158b9e6f8c35de67d9abbe
2017-08-22 19:50:00 -07:00
Jyothsna Konisa
43f27195b6 Modification of unloadChildrenNow
Summary: Modified `TreeInode::unloadChildrenNow` such that inodes are unloaded whose age is greater than a specific age.

Reviewed By: simpkins

Differential Revision: D5526137

fbshipit-source-id: 91e2364d55e31befedcf43d98c26467e1a472ef9
2017-08-18 14:20:43 -07:00
Jyothsna Konisa
916c129655 setting TimeStamps for TreeInode
Summary:
Updated time stamps of TreeInode accurately on mkdir,rmdir,mknode,symlink,create,unlink and readdir.
updated the `TreeInode::getattr` function to return in-memory timestamps.

Reviewed By: simpkins

Differential Revision: D5568183

fbshipit-source-id: c36f7fb767cd4342aab5cc983eea56e37cd2077e
2017-08-14 23:23:23 -07:00
Jyothsna Konisa
00bce5b020 implementing TreeInode::setInodeAttr
Summary:
Added `TreeInode::setInodeAttr` a helper function used in `InodeBase::setattr`. Also,added `InodeBase::setAtime` ,`InodeBase::setMtime`  and implemented them in `FileInode` and `TreeInode`.
Moved updating timestamp logic to `InodeBase::setattr` from `FileInode::setInodeAttr` and `TreeInode::setInodeAttr`.

Reviewed By: simpkins

Differential Revision: D5545422

fbshipit-source-id: 597cfabb3062166a058cf32776acb50a1bc0c61c
2017-08-14 13:36:37 -07:00
Wez Furlong
c08890f849 do a better job at reporting "new" in watchman results.
Summary:
We're seeing that this is always set to true for eden,
which is causing buck to run slower than it should.

To make this work correctly, I've augmented our journal data structure
so that it can track create, change and remove events for the various
paths.

I've also plumbed rename events into the journal.

This requires a slightly more complex merge routine, so I've refactored the two
call sites that were merging in slightly different contexts so that they can
now share the same guts of the merge routine.  Perhaps slightly
counterintuitive in the merge code is that we merge a record from the past into
the state for now and this is a bit backwards compared to how people think.

I've expanded the eden integration test to check that we don't mix up
create/change/removes for the same path in a given window.

On the watchman side, we use the presence of the filename in the createdPaths
set as a hint that the file is new.  In that case we will set the watchman
`ctime` (which is not the state ctime but is really the *created clock time*)
to match the current journal position if the file is new, or leave it set
to 0 if the file is not known to be new.  This will cause the `is_new`
flag to be set appropriately by the code in `watchman/query/eval.cpp`;
if the sequence is 0 then it should never be set to true.  Otherwise (when
the file was in the `createPaths` set) it will be set to the current journal
position and this will be seen as newer than the `since` constraint on
the query and cause the file to show as `new`.

Reviewed By: bolinfest

Differential Revision: D5608538

fbshipit-source-id: 8d78f7da05e5e53110108aca220c3a97794f8cc2
2017-08-11 12:57:37 -07:00
Jyothsna Konisa
3f046593a8 Wrapper for TimeStamps & helper function to set timestamps in setattr.
Summary:
1. Added a new structure `InodeBase::InodeTimestamps` to wrap atime,ctime,mtime together. This new structure helps in avoiding usage of `struct stat` for timestamps.
2. Modified function `Overlay::openFile` ,`Overlay::updateTimestampToHeader`, `Overlay::deserializeOverlayDir`, `Overlay::parseHeader` to use this new structure for timestamps instead of `struct stat`. Also, modified code in places where this change is being affected.
3. Added new helper methods `FileInode::setattrTimes`  and `TreeInode::setattrTimes` to set timestamps in FileInode and TreeInode during setattr. Implementation of setattr for FileInode and TreeInode is in the diffs stacked above this diff.
4. Replaced atime, ctime, mtime in `FileInode::State`, `TreeInode::Dir` to `FileInode::State::timeStamps` and `TreeInode::State::timeStamps`. Made other necessary changes to support this change.

Reviewed By: simpkins

Differential Revision: D5596854

fbshipit-source-id: 2786b7b695508a62fdf8f7829f1ce76054b61c52
2017-08-11 11:36:07 -07:00
Michael Bolin
48f3d2e5b9 Autoformat TreeInode.cpp before making other, unrelated changes.
Reviewed By: simpkins

Differential Revision: D5600811

fbshipit-source-id: e3d0e1da18937e2a3c7ebb77d730622ff62bbced
2017-08-10 14:55:04 -07:00
Jyothsna Konisa
6aa6e547d6 Reading and writing timestamps in to overlay files
Summary:
Added a new function `InodeBase::updateOverlayHeader` and implemented `FileInode::updateOverlayHeader` and `TreeInode::updateOverlayHeader` to update inmemory timestamps to overlay header when an inode is unreferenced.

Added helper functions in `Overlay` class to read and update timestamps in to the overlay file. Also,modified `Overlay::loadOverlayDir` to read and populate timestamps from overlay header in to treeinode.

Modified constructor of `FileInode::state` to read timestamps from overlay file and to populate inode timestamps.

Added test case to check if time stamps are updated and read correctly on remount.

Fixed a lint warning in TARGETS file

Reviewed By: simpkins

Differential Revision: D5535429

fbshipit-source-id: f6b758f70101c65d316a35101aacc9a3363f7aed
2017-08-04 20:19:20 -07:00
Jyothsna Konisa
6d2b510314 Fix in initializing last checkout time in Treeinode.
Summary: Changed the initialization of last checkout time in Tree inode constructor by grabbing a lock on `TreeInode::contents_`

Reviewed By: simpkins

Differential Revision: D5524857

fbshipit-source-id: e86ee7d986ca0c280ba156ba9146d6d1f9fa722e
2017-07-31 20:52:35 -07:00
Jyothsna Konisa
e4fefa3e69 Adding Timestamps to TreeInode class and intializing timestamps to lastcheckout time
Summary: Added atime,ctime,mtime for tracking timestamps for directories inmemory and initialized them to the last checkout time during the creation of TreeInode.

Reviewed By: bolinfest

Differential Revision: D5440950

fbshipit-source-id: 639cf1ce6722f80dde35d33849aa712aa30301a8
2017-07-27 18:25:01 -07:00
Jyothsna Konisa
20cd12b31a Moving FileData methods to FileInode
Summary:
Moved all the member functions from `FileData` class to `FileInode` class
and made `FileInode` methods independent of shared `FileData` object.
Removed `FileData.h` and `FileData.cpp` files as they are not needed anymore.

Modified functions `FileInode::getSHA1()` and `FileInode::isSameAsFast` and
modified few testcases which are currently using `FileData` class and made
sure that all the test cases are passing.

Reviewed By: bolinfest

Differential Revision: D5430128

fbshipit-source-id: 3e8e6c490e92e4e602355e4ce39b67c450ec53f8
2017-07-26 23:39:35 -07:00
Adam Simpkins
6f60e9318e fix crashes triggered by the new TreeInode::Entry checks
Summary:
D5483953 added a check to ensure that getMode() can only be called on entries
that do not have a loaded inode.  However, a few places in the code were still
calling getMode() on tree entries without checking if the inode was loaded or
not.

These crashes were caught in the integration tests run by `buck test eden/...`,
but were not caught by sandcastle tests on the original diff since sandcastle
only runs the eden unit tests, and not the integration tests.

All of these locations only needed to check the file type, which is safe to do
even if the inode is loaded, since the file type portion of the mode can never
change on an existing inode.  Only the permissions bits are unsafe to access
once an inode has been loaded (since we need to ask the inode itself for the
latest permissions bits).

I updated these call sites to stop using getMode() and instead use functions
that explicitly check only the file type bits.

Reviewed By: bolinfest

Differential Revision: D5501256

fbshipit-source-id: c989dab2fdacb5b9cdecb6f5101795298f57e78b
2017-07-26 13:21:13 -07:00
Saurabh Singh
55f92fe95d Making public fields of the TreeInode::Entry private
Summary:
Before this commit, TreeInode::Entry was a struct which had two
private members: mode and inode. In this commit,

  1. TreeInode::Entry was changed from struct to class.
  2. Appropriate getters and setters were introduced for the public members to
     make them private.
  3. Existing code accessing the public members directly was modified to use
     the getters instead.
  4. A couple of TODOs were added to address Overlay::saveOverlayDir()'s access
     of child inode information.

Reviewed By: simpkins

Differential Revision: D5483953

fbshipit-source-id: 50d526731e193a3a4a32742bd4d49deb9ee6b432
2017-07-25 20:12:59 -07:00
Victor Gao
a477e9663f comment out unused parameters
Summary: This uses `clang-tidy` to comment out unused parameters (in functions, methods and lambdas) in fbcode. Cases that the tool failed to handle are fixed manually.

Reviewed By: igorsugak

Differential Revision: D5454343

fbshipit-source-id: 5dee339b4334e25e963891b519a5aa81fbf627b2
2017-07-21 15:01:05 -07:00
Jyothsna Konisa
981d1d660a overlay header file related changes
Summary: Added header to the files that are materialized .

Reviewed By: simpkins

Differential Revision: D5387990

fbshipit-source-id: 1d551d674a39e01d6314d8bb3308e7bea3e669fc
2017-07-13 17:25:18 -07:00
Adam Simpkins
8be3b57eed fix issues updating TreeInode materialization status during checkout
Summary:
This updates the TreeInode code to remove the redundant materialized flag.
A TreeInode should have a Tree Hash if and only if it is dematerialized, so
there is no need for an extra `materialized` boolean.

This diff also fixes an issue in TreeInode::saveOverlayPostCheckout() where it
was not correctly informing it's parent TreeInode of the change if it moved
from one dematerialized state to another (with a different TreeInode hash).
This fixes the code to correctly call `parent->childDematerialized()` when it
needs to inform the parent that it now refers to a different source control
hash.

Reviewed By: wez

Differential Revision: D5336629

fbshipit-source-id: b4d86ecdef2f5faefbc243a09d869c02384ae95c
2017-07-07 18:45:02 -07:00
Jyothsna Konisa
299f7d1373 Integration Test for UnnloadFreeInodes
Summary:
1. Moved read, write, mkdir, rm methods in hg/lib/hg_extension_test_base.py to lib/test_case.py.
2. Added integration test case to test unload free inodes.

Reviewed By: simpkins

Differential Revision: D5277870

fbshipit-source-id: b93b6049a10357cf8c92366e6dca3968f7f30c30
2017-06-22 22:38:47 -07:00
Adam Simpkins
b0d1e57975 update logging statements to use folly logging APIs
Summary:
Update eden to log via the new folly logging APIs rather than with glog.

This adds a new --logging flag that takes a logging configuration string.
By default we set the log level to INFO for all eden logs, and WARNING for
everything else.  (I suspect we may eventually want to run with some
high-priority debug logs enabled for some or all of eden, but this seems like a
reasonable default to start with.)

Reviewed By: wez

Differential Revision: D5290783

fbshipit-source-id: 14183489c48c96613e2aca0f513bfa82fd9798c7
2017-06-22 13:50:13 -07:00
Adam Simpkins
6b62f186c7 drop Future from the name of ObjectStore APIs
Summary:
Now that the non-future versions of these APIs have been removed, rename
getBlobFuture() to getBlob(), and getTreeFuture() to getTree()

Reviewed By: wez

Differential Revision: D5295690

fbshipit-source-id: 30dcb88854b23160692b9cd83a632f863e07b491
2017-06-21 17:20:50 -07:00
Adam Simpkins
c3f47540b0 update the Dirstate to use TreeInode directly to unlink files
Summary:
Update Dirstate::remove() to directly call TreeInode::unlink() instead of going
through the EdenDispatcher.  Internal code should be able to use TreeInode
directly without going through the FUSE Dispatcher.  This prevents us from
having to look up the TreeInode object by inode number again.

This also fixes TreeInode::unlink() and TreeInode::rmdir() to perform FUSE
cache invalidation correctly.  EdenDispatcher used to do this for unlink()
calls, but it was missing for rmdir() calls.

Reviewed By: wez

Differential Revision: D4968832

fbshipit-source-id: 51fda5de196392c640436ca6ad7d8333e6799db9
2017-05-01 18:54:42 -07:00
Adam Simpkins
eb1a2b671c minor clean up to GitIgnoreStack handling
Summary:
Make sure we use pointers to const GitIgnoreStack objects when possible.

Also change the code to store the root GitIgnoreStack object for a diff()
operation in the DiffContext object, instead of having EdenMount::diff()
allocate it separately on the heap.  This will make it easier to consolidate a
bit more logic in the DiffContext class in the future.  In particular, I
suspect we will want a version of diff() that only works on one portion of the
tree.  Putting more of the functionality in DiffContext will make it slightly
easier to share code between the full-mount diff and the subtree diff
functions.  This also happens to save a memory allocation for now.

Reviewed By: wez

Differential Revision: D4968833

fbshipit-source-id: 1dc33b3d44cdf00e93b22d810c3a736d27c13638
2017-04-28 19:21:34 -07:00
Adam Simpkins
841c9006e3 honor trailing slashes on gitignore patterns
Summary:
Update the gitignore code so that patterns ending in a trailing slash correctly
match only directories.  Fortunately we have file type information available
during TreeInode::computeDiff(), so getting the correct file type information
does not add any extra overhead.  The older ignore checking code in
Dirstate.cpp can also do a reasonable job of getting file type information.
That code should also be removed at some point in the future, and use the
TreeInode logic for computing ignore status.

Reviewed By: bolinfest

Differential Revision: D4901326

fbshipit-source-id: 1222c8142876c91e1b80ec937ec84c0c28737224
2017-04-20 21:21:15 -07:00
Adam Simpkins
a6ae3edab9 move eden/utils and eden/fuse into eden/fs
Summary:
This change makes it so that all of the C++ code related to the edenfs daemon
is now contained in the eden/fs subdirectory.

Reviewed By: bolinfest, wez

Differential Revision: D4889053

fbshipit-source-id: d0bd4774cc0bdb5d1d6b6f47d716ecae52391f37
2017-04-14 11:39:02 -07:00