Commit Graph

507 Commits

Author SHA1 Message Date
Adam Simpkins
516539a9d4 re-enable treemanifest import by default
Summary:
Change the default value for --use_hg_tree_manifest to true, to re-enable
treemanifest import by default.  This should work much better now that we are
using the latest treemanifest code and now support fetching treemanifest data
from the remote server when it is not found locally.

Reviewed By: bolinfest

Differential Revision: D5647204

fbshipit-source-id: 424baf4de1d3b247c1e04a838040baeb5976404b
2017-08-17 16:13:00 -07:00
Adam Simpkins
094810c520 clean up treemanifest initialization in HgImporter
Summary:
This refactors the treemanifest union store initialization code, and fixes a
crash introduced in D5560787 if --use_hg_tree_manifest is enabled but the
underlying repository does not support treemanifest.

This moves the treemanifest initialization code into a new importTreeManifest()
helper function, and separates it from parsing the CMD_STARTED response.  We
now only initialize unionStore_ if FLAGS_use_hg_tree_manifest is enabled and
the repository supports treemanifest.  Other places in the code now only check
if unionStore_ is non-null to see if they should try importing via
treemanifest.

Splitting importTreeManifest() from the CMD_STARTED response parsing will
potentially make things cleaner in the future for using multiple
hg_import_helper.py processes.  Even if we start multiple of these processes,
we only need one instance of the C++ union store, and we don't need to
re-initialize the treemanifest separately each time.

Reviewed By: bolinfest

Differential Revision: D5647203

fbshipit-source-id: 71e14156f1d0ebc0880dd819c5b041b7c1146818
2017-08-17 16:13:00 -07:00
Adam Simpkins
ff1141cf5d fix journal merging of remove followed by create events
Summary:
A removal followed by a create should be treated as a change rather than a
no-op.  Previously the code was merging this into a no-op, and was also
ignoring any subsequent modifications in the same journal period being queried
(since it processes transactions in reverse order).

Reviewed By: bolinfest

Differential Revision: D5647669

fbshipit-source-id: 29389206f0e818d6b6248fbb6697b85e93064b1f
2017-08-17 12:33:51 -07:00
Andrew Gallagher
1275eaaa90 eden: disable autodeps on eden/fs/fuse/TARGETS
Summary:
Recently, autodeps switched to requiring simple types for the various dep
parameters in order to support proper annotation parsing (D5387184), which
started breaking on `eden/fs/fuse/TARGETS`.

Reviewed By: simpkins

Differential Revision: D5620249

fbshipit-source-id: 16fa2c73421ff7e9929c71290a662babde1289ec
2017-08-16 23:15:44 -07:00
Braden Watling
cf297e0106 Add subcommand getpath to eden debug
Summary: Add a command to lookup the path for an inode given the inode number and eden mount path.

Reviewed By: bolinfest

Differential Revision: D5627411

fbshipit-source-id: 25928f506d3f48d8a6784fe81fb17fa0500d6bc9
2017-08-16 20:04:30 -07:00
Adam Simpkins
322cad1e8a update the hg import tester to recursively import trees
Summary:
Update the hg import tester utility to recursively import all trees under the
specified path by default.  Passing in `--tree_import_recurse=no` can be used
to disable this behavior.

Reviewed By: wez

Differential Revision: D5588902

fbshipit-source-id: 1426dd18e0fb86728429f203b5be4742998886ed
2017-08-16 17:50:10 -07:00
Adam Simpkins
5137a50570 add a flag to disable fetching tree data from remote servers
Summary:
Add a flag to prevent HgImporter from trying to fetch missing tree data from
the remote mercurial server.

This is mainly useful for debugging purposes to force operations to fail if we
are missing some tree data locally.

Reviewed By: wez

Differential Revision: D5588900

fbshipit-source-id: b9f688383a0c1fceab614341473b357cda31a88b
2017-08-16 17:50:10 -07:00
Adam Simpkins
72345819c2 fix logging of the non-nul-terminated entry->flag field
Summary:
The `entry->flag` field in a ManifestEntry object is a `char*`, but it is not
actually a nul-terminated string.  When it is non-null, only the first
character should be used.  This updates the eden logging code to log just this
one character, instead of trying to log it like a normal `char*` and reading
garbage after the flag character.

Reviewed By: wez

Differential Revision: D5588899

fbshipit-source-id: 63b727a3aaee838f3a0cf6d765140964dd3ea746
2017-08-16 17:50:10 -07:00
Adam Simpkins
f9fef196f4 use the newer fb-hgext code from scm/hgext
Summary:
Summary
This removes the stale copy of the fb-hgext C/C++ code at eden/hg/datastorage.
The eden include paths have been updated to find the newer includes from
scm/hgext.

This will remove the treemanifest code from the eden-hg repository.  This code
is available in the fb-hgext bitbucket repository instead.  However, I will
likely send out a diff to `#ifdef` out the treemanifest integration from eden
in open source builds.  This will ensure that open source eden builds can
build without using the treemanifest code.

Reviewed By: wez

Differential Revision: D5588675

fbshipit-source-id: ab3f26c2797e95baaedeb9ca73fee56ddb521d6a
2017-08-15 12:15:06 -07:00
Jyothsna Konisa
916c129655 setting TimeStamps for TreeInode
Summary:
Updated time stamps of TreeInode accurately on mkdir,rmdir,mknode,symlink,create,unlink and readdir.
updated the `TreeInode::getattr` function to return in-memory timestamps.

Reviewed By: simpkins

Differential Revision: D5568183

fbshipit-source-id: c36f7fb767cd4342aab5cc983eea56e37cd2077e
2017-08-14 23:23:23 -07:00
Jyothsna Konisa
5c373f74c1 Removal of creation time from `FileInode::State
Summary: Removed creation time from `FileInode::state` which was used for getting timestamps of files that are not materialized.Now that we added timestamps to file inodes and tree inodes we no longer need this.

Reviewed By: wez

Differential Revision: D5552761

fbshipit-source-id: 6013b1f694045e08ada7bd64114c4f2e52848fef
2017-08-14 23:23:23 -07:00
Jyothsna Konisa
8bcd0f234c Setting TimeStamps for FileInode
Summary:
updating atime,ctime,mtime of FileInode on read, write and setattr system calls.
modified `FileInode::stat` function to return accurate inmemory timestamps.

Reviewed By: simpkins

Differential Revision: D5552666

fbshipit-source-id: 86d446f72908663f8db509b7b789d9f35d17df3a
2017-08-14 23:23:23 -07:00
Jyothsna Konisa
00bce5b020 implementing TreeInode::setInodeAttr
Summary:
Added `TreeInode::setInodeAttr` a helper function used in `InodeBase::setattr`. Also,added `InodeBase::setAtime` ,`InodeBase::setMtime`  and implemented them in `FileInode` and `TreeInode`.
Moved updating timestamp logic to `InodeBase::setattr` from `FileInode::setInodeAttr` and `TreeInode::setInodeAttr`.

Reviewed By: simpkins

Differential Revision: D5545422

fbshipit-source-id: 597cfabb3062166a058cf32776acb50a1bc0c61c
2017-08-14 13:36:37 -07:00
Jyothsna Konisa
7e230ba743 Implementing setattr in InodeBase (FileInode::setattr removal)
Summary: Removed `FileInode::setattr` from `FileInode`  and added a helper function `setInodeAttr` to perform FileInode or TreeInode specific setattr operations in `InodeBase::setattr`. This diff contains implementation of setattr for FileInode i.e for files, will add setattr implementation for directories in another diff.

Reviewed By: simpkins

Differential Revision: D5544968

fbshipit-source-id: 089491d07a603e111966987ef390b6e597aba28c
2017-08-14 13:36:37 -07:00
Christopher Dykes
0e3c7fe4e7 Shift everything out of folly/experimental:experimental
Summary: It wasn't worth putting these into separate diffs, so just do them all at once.

Reviewed By: yfeldblum

Differential Revision: D5619187

fbshipit-source-id: 7651c062601f4134fac4e4fea654e64d7d8cb8c8
2017-08-12 19:50:09 -07:00
Wez Furlong
c08890f849 do a better job at reporting "new" in watchman results.
Summary:
We're seeing that this is always set to true for eden,
which is causing buck to run slower than it should.

To make this work correctly, I've augmented our journal data structure
so that it can track create, change and remove events for the various
paths.

I've also plumbed rename events into the journal.

This requires a slightly more complex merge routine, so I've refactored the two
call sites that were merging in slightly different contexts so that they can
now share the same guts of the merge routine.  Perhaps slightly
counterintuitive in the merge code is that we merge a record from the past into
the state for now and this is a bit backwards compared to how people think.

I've expanded the eden integration test to check that we don't mix up
create/change/removes for the same path in a given window.

On the watchman side, we use the presence of the filename in the createdPaths
set as a hint that the file is new.  In that case we will set the watchman
`ctime` (which is not the state ctime but is really the *created clock time*)
to match the current journal position if the file is new, or leave it set
to 0 if the file is not known to be new.  This will cause the `is_new`
flag to be set appropriately by the code in `watchman/query/eval.cpp`;
if the sequence is 0 then it should never be set to true.  Otherwise (when
the file was in the `createPaths` set) it will be set to the current journal
position and this will be seen as newer than the `since` constraint on
the query and cause the file to show as `new`.

Reviewed By: bolinfest

Differential Revision: D5608538

fbshipit-source-id: 8d78f7da05e5e53110108aca220c3a97794f8cc2
2017-08-11 12:57:37 -07:00
Jyothsna Konisa
3f046593a8 Wrapper for TimeStamps & helper function to set timestamps in setattr.
Summary:
1. Added a new structure `InodeBase::InodeTimestamps` to wrap atime,ctime,mtime together. This new structure helps in avoiding usage of `struct stat` for timestamps.
2. Modified function `Overlay::openFile` ,`Overlay::updateTimestampToHeader`, `Overlay::deserializeOverlayDir`, `Overlay::parseHeader` to use this new structure for timestamps instead of `struct stat`. Also, modified code in places where this change is being affected.
3. Added new helper methods `FileInode::setattrTimes`  and `TreeInode::setattrTimes` to set timestamps in FileInode and TreeInode during setattr. Implementation of setattr for FileInode and TreeInode is in the diffs stacked above this diff.
4. Replaced atime, ctime, mtime in `FileInode::State`, `TreeInode::Dir` to `FileInode::State::timeStamps` and `TreeInode::State::timeStamps`. Made other necessary changes to support this change.

Reviewed By: simpkins

Differential Revision: D5596854

fbshipit-source-id: 2786b7b695508a62fdf8f7829f1ce76054b61c52
2017-08-11 11:36:07 -07:00
Michael Bolin
48f3d2e5b9 Autoformat TreeInode.cpp before making other, unrelated changes.
Reviewed By: simpkins

Differential Revision: D5600811

fbshipit-source-id: e3d0e1da18937e2a3c7ebb77d730622ff62bbced
2017-08-10 14:55:04 -07:00
Wez Furlong
39b21f383b compensate for the new header when debugging the overlay
Summary:
I'm trying to troubleshoot why a new dir that I added isn't showing up in the overlay and I found that the debug tool doesn't know about the overlay.

This is the dumbest minimal thing I could do to make this basically functional again

Reviewed By: simpkins

Differential Revision: D5602439

fbshipit-source-id: 4ae7fa34136697f9f915ccd95275cb3a7923ae4a
2017-08-10 11:42:25 -07:00
Jyothsna Konisa
20c62ae2bf Refactoring FileInode::setattr
Summary: Currently we have two functions `FileInode::setattr` and `FileInode::setAttr` which are used to set given attributes to a `FileInode`. Merged both the functions in to one function called `FileInode::setattr` and removed `FileInode::setAttr`.

Reviewed By: wez

Differential Revision: D5538490

fbshipit-source-id: ec241fad25d6e4694865e5fc3c0a3500e4838bdd
2017-08-04 20:19:20 -07:00
Jyothsna Konisa
6aa6e547d6 Reading and writing timestamps in to overlay files
Summary:
Added a new function `InodeBase::updateOverlayHeader` and implemented `FileInode::updateOverlayHeader` and `TreeInode::updateOverlayHeader` to update inmemory timestamps to overlay header when an inode is unreferenced.

Added helper functions in `Overlay` class to read and update timestamps in to the overlay file. Also,modified `Overlay::loadOverlayDir` to read and populate timestamps from overlay header in to treeinode.

Modified constructor of `FileInode::state` to read timestamps from overlay file and to populate inode timestamps.

Added test case to check if time stamps are updated and read correctly on remount.

Fixed a lint warning in TARGETS file

Reviewed By: simpkins

Differential Revision: D5535429

fbshipit-source-id: f6b758f70101c65d316a35101aacc9a3363f7aed
2017-08-04 20:19:20 -07:00
Adam Simpkins
56230b362c add additional info to the hg_import_helper "started" response
Summary:
Update the hg_import_helper.py script to include additional information in the
initial CMD_STARTED response that it sends after it has successfully
initialized.

This response now includes:
- A protocol version number, to help catch errors if edenfs somehow ends up
  invoking an incompatible version of the hg_import_helper.py script.
- Whether treemanifest is supported in this repository.
- If treemanifest is supported, the list of treemanifest pack directories.

This eliminates the need for the separate GET_CACHE_PATH command that was
previously sent immediately after the CMD_STARTED response.

This also fixes the code to always let the python code select the pack
directories.  Previously the C++ code assumed the local pack directory was
aways located at ".hg/store/packs/manifest" under the repoPath.  This may not
be correct if repoPath happens to be a repository created with the share
extension.

Reviewed By: wez

Differential Revision: D5560787

fbshipit-source-id: e796222f42927a85e886227b3a7b2ccf9c1ef1bd
2017-08-04 18:46:54 -07:00
Adam Simpkins
0285670135 fetch missing tree data from a remote hg server when needed
Summary:
This is a first pass at updating eden to fetch tree manifest data from a remote
mercurial server on demand.  The code now catches MissingKeyErrors thrown by
the datastore code, and then fetches the requested tree data from the remote
server.

This makes it safe to use tree manifest for import, as we can now fetch missing
tree data as needed, instead of always returning I/O errors whenever we reach a
tree not available locally.

Unfortunately, the performance of downloading tree data is currently not very
good: it takes 90+ seconds plus to download any tree data.  The current
mercurial server code appears to always provide full recursive tree data
(unless you ask for the delta between two manifests, which we do not have).
Even when asking for very small subdirectories, the server appears to send the
full tree manifest data for the entire repository, sometimes taking longer than
if we had asked for the full repository data to start with.

Note that tree manifest import is still disabled by default with this diff, and
must be explicitly enabled by running edenfs with `--use_hg_tree_manifest`.

Reviewed By: wez

Differential Revision: D5544817

fbshipit-source-id: 940e0c914f055edc1beee438b0ac50c2f8b08b03
2017-08-04 18:46:54 -07:00
Adam Simpkins
9f1bb674f7 support importing subdirectory trees in the import tester
Summary:
Add logic to the hg import tester tool to support recursively importing
specific subdirectories using the tree manifest import mechanism.

Reviewed By: wez

Differential Revision: D5544818

fbshipit-source-id: 1637d32691c30dfab8d59599891feab6fb27bcdb
2017-08-03 18:00:27 -07:00
Adam Simpkins
497a2ae765 update hg import tester to help benchmarking
Summary:
This makes several improvements to the hg import tester script:

- If no --edenDir flag is specified, initialize a new temporary directory to
  keep the RocksDB data store.

- Add a `--rocksdb_options_file` flag to allow controlling the options used for
  the RocksDB store.

- Add an --import_type flag to allow explicitly selecting if we should test
  the flat manifest or tree manifest import code.

- Add a --flat_import_file flag, to allow testing a pre-generated flat manifest
  input data file, rather than retrieving the data from mercurial.  This allows
  benchmarking only the C++ import code, and eliminating the python portion of
  the import.  The input file can be generated by running
  `hg_import_helper.py --manifest <revision>`

Reviewed By: wez

Differential Revision: D5541732

fbshipit-source-id: 340af4fea872412248d41453792b2179f0afa466
2017-08-03 18:00:22 -07:00
Adam Simpkins
66ed1cd727 use folly::Synchronized to manage the mount point map
Summary:
Update EdenServer to use folly::Synchronized for managing access to the
mountPoints_ map, instead of using a separate standalone mutex.  This simply
makes it slightly harder to accidentally access the mount point map without
holding the lock correctly.

Reviewed By: bolinfest, wez

Differential Revision: D5541315

fbshipit-source-id: 094a941a3b8177aed4a75b91a998494f4c33fe8c
2017-08-02 17:42:49 -07:00
Adam Simpkins
0738182ea7 allow specifying --config options to hg_import_helper.py
Summary:
Update `hg_import_helper.py` to take `--config` options to set mercurial
options, just like the normal `hg` command.

This makes it easier to override mercurial settings during testing.  For
instance, this lets us override the `remotefilelog.cachepath` option for
testing tree manifest import without hitting the real system hgcache
directory.

Reviewed By: bolinfest

Differential Revision: D5523181

fbshipit-source-id: 8bb6f5f244fa979872275cbb6ee797d1477a9b5b
2017-08-02 17:07:19 -07:00
Adam Simpkins
4949aada7f fix EdenServer::unmount() to fully wait for mount point cleanup
Summary:
This fixes EdenServer::unmount() to actually wait for all EdenMount cleanup
to complete, and fixes unmountAll() to return a Future that correctly waits for
all mount points to be cleaned up.

Previously `unmount()` waited for the mount point to be unmounted from the
kernel, but did not wait for EdenMount shutdown to complete.  Previously
EdenMount shutdown was not triggered until the last reference to the
shared_ptr<EdenMount> was released.  This often happened in the FUSE channel
thread that triggered the mountFinished() call--it would still hold a
reference to this pointer, and would not release it until after
mountFinished() returns.  As a result, when the main thread was shutting down,
`main()` would call `unmountAll()`, and then return soon after it completed.
Some FUSE channel threads may still be running at this point, still performing
`EdenMount` shutdown while the main thread was exiting.  This could result in
crashes and deadlocks as shutdown tried to access objects already destroyed by
the main thread.

With this change `EdenMount::shutdown()` is triggered explicitly during
`mountFinished()`, and `unmount()` will not complete until this finishes.
The `EdenMount` object may still exist at this point, and could still be
deleted by the FUSE channel thread, but the deletion now only requires freeing
the memory and does not require accessing other data that may have been cleaned
up by the main thread.

We should still clean up the FUSE channel thread handling in the future, to
make sure these threads are joined before the main thread exits.  However, that
cleanup can wait until a separate diff.  Ideally I would like to move more of
the mount and unmount logic from EdenServer and EdenServiceHandler and put that
code in EdenMount instead.

Reviewed By: bolinfest

Differential Revision: D5541318

fbshipit-source-id: 470332478357a85c314bc40458373cb0f827f62b
2017-08-02 17:07:19 -07:00
Adam Simpkins
7a7e32090f wait for the unload inodes function to finish when unmounting
Summary:
When cleaning up an unmounted EdenMount, cancel the periodic unload inodes
function before we remove the mount from the mountPoints_ map and signal other
threads that it has been removed via mountPointsCV_.  Also use
FunctionScheduler::cancelAndWait() to make sure that we wait for the function
to complete if it was currently in the middle of being run.

Reviewed By: bolinfest

Differential Revision: D5541316

fbshipit-source-id: 46046f59746b5f283a5d4af94159b5c122d43d0d
2017-08-01 20:53:09 -07:00
Eddie Elizondo
789fa65712 Compiler: Enable partial use of mstch-cpp2
Summary:
Make all the changes in thrift to enable the use of mstch-cpp2.

For now, if any cpp2 flag is present, it will fallback to the python genenerator (i.e. t_cpp_generator.py)

Reviewed By: yfeldblum

Differential Revision: D5464216

fbshipit-source-id: 287618fbf3b2dcc3142e305e648e48b1ebb762c1
2017-08-01 09:21:38 -07:00
Jyothsna Konisa
6d2b510314 Fix in initializing last checkout time in Treeinode.
Summary: Changed the initialization of last checkout time in Tree inode constructor by grabbing a lock on `TreeInode::contents_`

Reviewed By: simpkins

Differential Revision: D5524857

fbshipit-source-id: e86ee7d986ca0c280ba156ba9146d6d1f9fa722e
2017-07-31 20:52:35 -07:00
Jyothsna Konisa
638615b542 periodic job to unload inodes
Summary:
Added `EdenServer::unloadInodesScheduler_` to schedule unloading inodes on timely basis.
new function will be added to the function scheduler on each mount, so each mount point can be have different start delay and different frequency.

Reviewed By: simpkins

Differential Revision: D5501036

fbshipit-source-id: 238b9cb5747a2358c65a1508095ad672bdf87ffc
2017-07-31 20:52:35 -07:00
Jyothsna Konisa
f0587420a2 Race condition Fix on overlay lock during unmount
Summary:
Some test cases are crashing once in a whiled due to a race condition on lock over overlay directory.

when a mount point is unmounted, `EdenServer::mountFinished` called on `umount` where destruction of `EdenMount` object for the mount point happens. When a remount is called before the destruction of `EdenMount` a race condition is occuring on the lock of overlay directory as the undestructed `EdenMount` still holds the lock on overlay and the newly created `EdenMount` tries to grab a lock on overly.

Made the `EdenServer::unmount` asynchronous which waits for a promise that is set in `EdenServer::mountFinished`.

Reviewed By: simpkins

Differential Revision: D5515740

fbshipit-source-id: 5cbfddf13fe00c2cb1b63a6be6353e9b84fbd569
2017-07-31 20:52:35 -07:00
Michael Bolin
50191d5bbd Fix a race condition in FileInode::readAll().
Summary:
In testing, we discovered a case where concurrent Hg operations in EdenFS would
//sometimes// fail with `ECONNREFUSED` when trying to read
`<eden-mount>/.eden/socket`.

This was very confusing, as the standard reasons for `ECONNREFUSED` did not seem
to apply:
- We verified that Eden had not crashed.
- We verified that Eden's UNIX domain socket had a sufficiently large backlog
  (1024) such that we should not be at risk of exhausting it with two simultaneous
  Hg commands.

It turned out that there was a race condition in our `readlink()` command, which
could cause `<eden-mount>/.eden/socket` to resolve to the wrong path. Failing to
connect to this path manifested itself as an `ECONNREFUSED` error.

It turned out that `readlink()` used `FileInode::readAll()`, which was
performing an `lseek()` followed by a `read()` on a file descriptor while the
descriptor was protected by a //read lock// instead of a //write lock//. Because
the `lseek()` causes a state change, it needs to be protected by a write lock.
Changing the type of the lock fixed the issue.

The only other caller of `FileInode::readAll()` appears to be in
`TreeInode::loadGitIgnoreThenDiff()`, so presumably this fixes a possible race
condition there, as well.

Reviewed By: simpkins

Differential Revision: D5533001

fbshipit-source-id: 86cf84c45463b2ae194d6f46909ea67c0f71d065
2017-07-31 19:08:09 -07:00
Adam Simpkins
10be70e0a3 minor cleanups to the strace-style logging
Summary:
Make a few minor tweaks to the strace-style logging added in D5464387.

- Call the log categories "eden.strace.<mount_path>" instead of
  "eden/strace<mount_path>".  The folly logging library uses '.' to separate
  nodes in the log category hierarchy, so this puts all of the strace messages
  under the "eden.strace" category, which it itself part of the "eden"
  category.  The "<mount_path>" will contain slashes inside it, but slashes are
  not treated specially in log category names.

- Rename `EdenMount::getLogger()` to `EdenMount::getStraceLogger()` since this
  logger should be used only for strace-style events, and not for general log
  messages for this mount point.

Reviewed By: bolinfest

Differential Revision: D5515245

fbshipit-source-id: 9d833d9fbff47c6a57a7afefeae92755ff0e28b7
2017-07-27 20:20:39 -07:00
Adam Simpkins
47ba88f62d move dropPrivileges() to UserInfo, and restore supplimentary groups
Summary:
Move the dropPrivileges() function from the PrivHelper code to the UserInfo
class, and update it to correctly set supplementary groups when dropping
privileges.

Reviewed By: bolinfest

Differential Revision: D5501254

fbshipit-source-id: 62d7a91685ae72a73e848236f6e6f636b6203481
2017-07-27 19:39:00 -07:00
Adam Simpkins
61cac1cbea add a new UserInfo class for looking up UID/GID info
Summary:
This adds a new UserInfo class to store the UID, GID, as well as the username
and home directory.  This moves the determineUid() and determineGid() functions
from main.cpp into this new class, and makes the logic somewhat smarter now.

In addition to looking up the UID and GID, we now look up the username.  This
information is not used yet, but will be used in an upcoming diff to set
supplementary groups.

This also stores the home directory in the UserInfo class.  The home directory
is usually necessary to find the user's ~/.edenrc file.  Computing it as part
of UserInfo makes the most sense since we will likely have already looked up
the user's passwd entry.

Reviewed By: bolinfest

Differential Revision: D5501252

fbshipit-source-id: 1cb4be9f6c1493de4362da3393034e78bedd9db2
2017-07-27 19:38:59 -07:00
Jyothsna Konisa
e4fefa3e69 Adding Timestamps to TreeInode class and intializing timestamps to lastcheckout time
Summary: Added atime,ctime,mtime for tracking timestamps for directories inmemory and initialized them to the last checkout time during the creation of TreeInode.

Reviewed By: bolinfest

Differential Revision: D5440950

fbshipit-source-id: 639cf1ce6722f80dde35d33849aa712aa30301a8
2017-07-27 18:25:01 -07:00
Michael Bolin
c8faca5285 Updated RebaseTest to verify that Eden's fast path was exercised.
Summary:
We verify this behavior by adding the appropriate debug statements to our Hg
extension, running `hg update` with the `--debug` flag, and verifying the
expected log statements appear in stdout.

Reviewed By: simpkins

Differential Revision: D5505478

fbshipit-source-id: 389b69c7c1de4fa88fd07fb9eb4dd7e3370e766b
2017-07-27 17:24:01 -07:00
Michael Bolin
31eac649e2 Add integration test for rebasing in Hg.
Summary:
This is a relatively simple test that takes two branches with no conflicts and
rebases one on top of the other. It also provides modest checks to ensure Eden
does not load a bunch of inodes unnecessarily when updating to the new head.

This also introduces `EdenServerInspector`, which provides convenience methods
for inspecting the Eden server via Thrift.

Reviewed By: simpkins

Differential Revision: D5504741

fbshipit-source-id: 6636c431658f24a850d0e5404d1a0e4f0528a781
2017-07-27 17:24:01 -07:00
Michael Bolin
b9b9ba32e9 Implement savebackup() and restorebackup() in eden_dirstate.
Summary:
Up until now, we have largely ignored Mercurial's logic around transactions.
Previously, the `savebackup()` and `restorebackup()` methods of `eden_dirstate`
were no-ops, but with this change, we provide real implementations for
`savebackup()` and `restorebackup()`.

We no longer try to maintain the `_dirty` field in `eden_dirstate` as of this change
as we should not be exercising any superclass code that relies on its value.

Reviewed By: simpkins

Differential Revision: D5485950

fbshipit-source-id: effa5f83ead8c6eefa613c196d7ec5646da31e4b
2017-07-27 17:24:01 -07:00
Adam Simpkins
05288c2b98 put PathFuncs.h in their own library rule
Summary:
This updates the eden/fs/utils/TARGETS file so that PathFuncs.h and
PathFuncs.cpp are in their own separate library.  The existing "utils" library
depends on it, but other users can now depend on PathFuncs.h without pulling in
the other code in utils.

I plan to make some of the privhelper code depend on PathFuncs.h in an upcoming
diff, and this makes it so that privhelper will not need to pull in the rest of
the utils code.

Reviewed By: bolinfest

Differential Revision: D5501253

fbshipit-source-id: 804067785ed663e223977d1c84ebcfc28b3f1642
2017-07-27 13:42:45 -07:00
Jyothsna Konisa
19df19d994 Adding lastCheckoutTime to EdenMount and initializing timestamps of FileInode with lastCheckoutTime
Summary:
Added a new data member lastCheckoutTime to EdenMount class to store the time when checkout operation is performed. Also added a method to get the last checkout time which internally returns the lastCheckoutTime in EdenMount class.

Added new fields atime,mtime,ctime in FileInode::State structure to keep track of timestamps in memory. Initialzed timestamps in FileInode::State constructor by calling getLastCheckOutTime from EdenMount class.

Still have to add timestamp tracking for directories and have to initialize them with lastCheckout time.This probably will be done in a seperate diff.

Reviewed By: bolinfest

Differential Revision: D5437682

fbshipit-source-id: e3d6b1bc0c2192538dd3b0d9a6017ceb3ca0843d
2017-07-27 11:52:31 -07:00
Jyothsna Konisa
20cd12b31a Moving FileData methods to FileInode
Summary:
Moved all the member functions from `FileData` class to `FileInode` class
and made `FileInode` methods independent of shared `FileData` object.
Removed `FileData.h` and `FileData.cpp` files as they are not needed anymore.

Modified functions `FileInode::getSHA1()` and `FileInode::isSameAsFast` and
modified few testcases which are currently using `FileData` class and made
sure that all the test cases are passing.

Reviewed By: bolinfest

Differential Revision: D5430128

fbshipit-source-id: 3e8e6c490e92e4e602355e4ce39b67c450ec53f8
2017-07-26 23:39:35 -07:00
Michael Bolin
f91e7b07ec Created integration test for hg rm.
Summary:
Added an integration test to verify behavior for `hg rm`. The test passed
without any changes to Eden required!

(Note: this ignores all push blocking failures!)

Reviewed By: simpkins

Differential Revision: D5494421

fbshipit-source-id: ca9578268adb353380e96aced6e2f1caca52b6a3
2017-07-26 22:08:09 -07:00
Adam Simpkins
6f60e9318e fix crashes triggered by the new TreeInode::Entry checks
Summary:
D5483953 added a check to ensure that getMode() can only be called on entries
that do not have a loaded inode.  However, a few places in the code were still
calling getMode() on tree entries without checking if the inode was loaded or
not.

These crashes were caught in the integration tests run by `buck test eden/...`,
but were not caught by sandcastle tests on the original diff since sandcastle
only runs the eden unit tests, and not the integration tests.

All of these locations only needed to check the file type, which is safe to do
even if the inode is loaded, since the file type portion of the mode can never
change on an existing inode.  Only the permissions bits are unsafe to access
once an inode has been loaded (since we need to ask the inode itself for the
latest permissions bits).

I updated these call sites to stop using getMode() and instead use functions
that explicitly check only the file type bits.

Reviewed By: bolinfest

Differential Revision: D5501256

fbshipit-source-id: c989dab2fdacb5b9cdecb6f5101795298f57e78b
2017-07-26 13:21:13 -07:00
Saurabh Singh
55f92fe95d Making public fields of the TreeInode::Entry private
Summary:
Before this commit, TreeInode::Entry was a struct which had two
private members: mode and inode. In this commit,

  1. TreeInode::Entry was changed from struct to class.
  2. Appropriate getters and setters were introduced for the public members to
     make them private.
  3. Existing code accessing the public members directly was modified to use
     the getters instead.
  4. A couple of TODOs were added to address Overlay::saveOverlayDir()'s access
     of child inode information.

Reviewed By: simpkins

Differential Revision: D5483953

fbshipit-source-id: 50d526731e193a3a4a32742bd4d49deb9ee6b432
2017-07-25 20:12:59 -07:00
Michael Bolin
401c69760d Pass a proper opener object to the parent constructor in eden_dirstate.
Summary:
Now `eden_dirstate` can inherit the implementations of the `branch()` and
`setbranch()` methods.

More importantly, we have decided to let `dirstate` do its natural management
of the `.hg` directory (with some exceptions, such as the contents of the
`.hg/dirstate` file) because there are interactions with `localrepo` and other
parts of Mercurial that use the filesystem as the API rather than an explicit
programmatic API, and we want to make sure we maintain the expected invariants.

(Note: this ignores all push blocking failures!)

Reviewed By: simpkins

Differential Revision: D5464059

fbshipit-source-id: 539e142ba11caea3e8487a3cc70d53cc236d2545
2017-07-25 11:58:26 -07:00
Victor Gao
a477e9663f comment out unused parameters
Summary: This uses `clang-tidy` to comment out unused parameters (in functions, methods and lambdas) in fbcode. Cases that the tool failed to handle are fixed manually.

Reviewed By: igorsugak

Differential Revision: D5454343

fbshipit-source-id: 5dee339b4334e25e963891b519a5aa81fbf627b2
2017-07-21 15:01:05 -07:00
Eamonn Kent
6141d8ba20 Use FB_LOG in order to allow category logging (by mount path)
Summary:
Log messages to an eden.strace category.
This allows us to enable/disable based on the mount_path. For example:

./buck-out/gen/eden/cli/cli.par daemon -F -- --logging  eden/strace/data/users/ekent/eden-NEW=DBG7

Thus, we are using a category, rather than filename (default)

Reviewed By: bolinfest

Differential Revision: D5464387

fbshipit-source-id: 6a54badd6bb806cfcda1742fd970073d91303396
2017-07-21 06:54:24 -07:00