Summary:
setattr is a bit of a multi-purpose interface; depending on the flag
values, this is responsible for:
* ftruncate(2)
* fchmod(2)
* fchown(2)
* futimens(2)
In order to apply any of these things, we have to materialize the file. In
the future we may want to allow setting the utimes without materializing the file.
We don't allow chown to actually chown anything. We may want to relax that in
the future, but at the moment we will return an error if an attempt is made to
change the ownership of a file.
Reviewed By: bolinfest
Differential Revision: D3511011
fbshipit-source-id: 858d2c07686fcbe2dcdb60a07527f739a9726be3
Summary:
Make sure mount points are completely stopped before destroying the EdenServer
object. Previously the EdenServer was destroyed with the MountPoints still
running the fuse channels in background threads. When the privileged helper
process unmounted them, fuse requests from the kernel could arrive and access
memory that had already been destroyed.
Reviewed By: wez
Differential Revision: D3458898
fbshipit-source-id: 365bca716ff0f8315b66af92effeb8c6dc574ce1
Summary:
When running the integration tests when built with gcc, the tests would crash
in TreeInode::create(). It appears that the unique_ptr<FileHandle> object was
getting passed to the lambda before dereferencing it to call getattr().
Reviewed By: wez
Differential Revision: D3459605
fbshipit-source-id: 5e2ce98d268a85731acaf7d7f37f22c77fb571cf
Summary:
This fixes inode number handling problems in TreeInode and TreeEntryFileInode.
Previosly these classes each had an ino_ member variable, despite deriving from
InodeBase which has its own (private) ino_ member.
TreeEntryFileInode never actually initialized its own local ino_ variable.
This was causing problems for many applications which use inode numbers to
cache file data. TreeEntryFileInode returned garbage data in the inode field,
resulting in incorrect cache collisions.
This fixes the inode handling, and also fixes the stat data returned by
FileData. It now sets the uid, gid, and inode fields correctly. It 0s out
several other fields rather than leaving them uninitialized.
Reviewed By: bolinfest
Differential Revision: D3455126
fbshipit-source-id: 631276b01676733f96035bc153219ef84406dcc9
Summary:
Start adding utility code to make it easier for the integration tests to
manipulate git and mercurial repositories.
This adds an HgRepository class, but does not update the tests to use it yet.
I will do that in a separate diff.
Reviewed By: bolinfest
Differential Revision: D3449328
fbshipit-source-id: caea5befb683a388cbad30a29017c566e22702af
Summary:
Move the eden/fs/importer/hg code to eden/fs/store/hg. This import code is
only used by the HgBackingStore, so logically it belongs together.
The eden/fs/importer/git directory was already moved into eden/fs/store/git by
D3448752, so this makes the mercurial code similar.
Reviewed By: bolinfest
Differential Revision: D3458128
fbshipit-source-id: 54b1f707197e41ff5f5f4a163a34d3d1ab432e21
Summary:
This moves all of the test library code into a lib/ subdirectory, just to help
distinguish tests from utility code.
This also changes the test so that we no longer pack the eden CLI and daemon
binaries into the python archives. This results in very large archives when
building in dbg and opt modes, and isn't really necessary. Instead
edenclient.py simply finds the CLI and daemon binaries relative to the test
binary. We pass in an EDENFS_SUFFIX variable to tell it which flavor of the
daemon to use.
Additionally, this changes the tests to run with python 3.
Reviewed By: bolinfest
Differential Revision: D3449013
fbshipit-source-id: 82533137090325766a52cd067aa97dd8391ae088
Summary:
This moves git import logic from the GitImporter class to GitBackingStore.
The logic is simpler now, since GitBackingStore only needs to import a single
Tree or Blob at a time.
Reviewed By: bolinfest
Differential Revision: D3448752
fbshipit-source-id: da2d59f953ada714d8512545ae83dd48e5d3e410
Summary:
Add a run_cmd() function that's slightly simpler to use compared to
_get_eden_args().
Reviewed By: bolinfest
Differential Revision: D3448844
fbshipit-source-id: 06b654b2f4bc773d67a4f9d1f6effdad2ce9aa2a
Summary:
- Update "eden daemon" to wait until edenfs is healthy before returning.
- Accept additional command line arguments and pass them through to edenfs.
- When starting edenfs as a background daemon, direct stdout and stderr to a
log file in the eden config directory.
- Update "eden shutdown" command to wait for edenfs to exit.
- Update "eden shutdown" command to print an error message if edenfs was not
running.
Reviewed By: bolinfest
Differential Revision: D3446403
fbshipit-source-id: 2ca1874256b7d124a85092b886548bea8c198327
Summary:
Look for hg_import_helper.py relative to the edenfs binary, rather than
relative to the current working directory. Also check in a couple places, so
we can work in both normal deployment locations as well as inside a source
repository.
Additionally, update hg_import_helper.py to emit a response chunk as soon as it
starts. This contains either message indicating that it has started
successfully, or an error message. This allows us to propagate error
information back to the thrift caller if something goes wrong starting
hg_import_helper.py for a given repository.
Reviewed By: bolinfest
Differential Revision: D3447027
fbshipit-source-id: 4bffd1d03ab6475c09cb3c8385145555c7fc7361
Summary:
[Folly] Wrappers for some of OpenSSL's crypto hash functions.
Wraps some of the OpenSSL crypto hash functions with variants that take `ByteRange` for input and `MutableByteRange` for output, and also variants that take `const IOBuf&` for input as well.
These are a bit nicer to use than passing pointers and lengths separately.
Reviewed By: ivmaykov
Differential Revision: D3434562
fbshipit-source-id: 3688ef11680a029b7664ac417a7781e70f9c6926
Summary:
This adds a --foreground flag to the "eden daemon" command, which causes the
daemon to run in the foreground, rather than daemonizing as a background
process. Additionally, this drops the --preserve-environment flag and instead
updates the CLI to always build a sane environment to run with.
This also updates the integration test code to use this flag, and wait for eden
to shut down during test clean up. Without this, the test case code could try
to clean up temporary directories before they were unmounted, which would fail.
This would leave many temporary directories behind after test runs.
This also re-organizes the temporary directory set up a bit, so that each test
uses a single top-level temporary directory. All other directories it creates
are put inside this one directory.
Reviewed By: bolinfest
Differential Revision: D3439232
fbshipit-source-id: 85305f7db60da6bae589d28c802ee260e42e4dea
Summary:
This changes the way that Eden is built and deployed.
* To build the binary that must be run as `root` (but quickly drops privileges), run `buck build eden-daemon`.
* To build the CLI that communicates with the daemon (and does not require privileges), run `buck build eden-cli`.
* To build both, run `buck build eden`.
There is an example of how to build the various parts of Eden using
Buck and how to package them up in the `install` script introduced by this revision.
While here, I also cleaned up some of our build files and changed them to be
parameterized between internal and external use. In both cases, the user gets the
"unadorned" version of their primary build targets. This ensures that shortcuts such as:
```
buck test eden/fs/integration
```
do the right thing by default.
Finally, I also made `find_default_config_dir()` and `find_default_daemon_binary()`
lazy whereas `find_default_config_dir()` was previously eager.
Reviewed By: simpkins
Differential Revision: D3436245
fbshipit-source-id: 4dfbd59ed0d198620324f0705c462334bb5a7daf
Summary:
If a directory is present in the overlay, we still need to check if a TreeEntry
exists from the source control data structures. Previously this was causing us
to incorrectly report directories as empty if they exist in the local overlay,
even when they had contents from the main Tree.
Reviewed By: wez
Differential Revision: D3434219
fbshipit-source-id: f872f90075602dfdc7b217f50eefcd7c248512e7
Summary:
Add some verbose logging about when trees and blobs are loaded in the object
store.
Reviewed By: bolinfest
Differential Revision: D3434182
fbshipit-source-id: 3e8d2617290604f119e6164d15d63324a4c9a2aa
Summary:
Update the HgImporter class to support retrieving file contents from mercurial.
This also includes simple code for storing the data in the LocalStore using
git's blob serialization format. In the future I think it would perhaps be
better to drop the "blob<length>" prefix, and instead just use a RocksDB column
family to separate blob data from other types of data. However, for now using
the git format is simplest for keeping compatibility with the getBlob() code.
Reviewed By: bolinfest
Differential Revision: D3416691
fbshipit-source-id: 268787533be2172b2dbedc3bf06464eabf3d2c5e
Summary:
Record mercurial commits that have been previously imported, and avoid
re-importing them when checking out a commit that was previously processed and
already has a tree present in the LocalStore.
Reviewed By: bolinfest
Differential Revision: D3416654
fbshipit-source-id: 80f70fa2204d3a2961991b634ad934a623b42b82
Summary:
Add APIs for storing arbitrary (key, value) data.
This will allow BackingStore implementations to store additional metadata, such
as mapping mercurial commit IDs to the eden root tree ID.
Eventually we may want to use RocksDB column families to partition the
different types of data being put into the LocalStore. However, for now this
just uses a single key space. We can add column family support in a separate
diff, if desired.
Reviewed By: bolinfest
Differential Revision: D3409866
fbshipit-source-id: 19a1d340b65bff2081981bf5daf32d5ad15b60c4
Summary:
This adds an HgBackingStore implementation which can load tree data from a
mercurial repository. Blob loading is not implemented yet, but will come in a
separate diff.
This also adds a minimal GitBackingStore class. The GitBackingStore has nearly
no functionality, but is needed to keep the existing git functionality working.
Reviewed By: bolinfest
Differential Revision: D3409743
fbshipit-source-id: dbebf53e9de08bd1469e489baa48b84cbf889511
Summary:
Add the basic BackingStore interface, plus a NullBackingStore implementation
that always returns null. This updates the ObjectStore to query the
BackingStore if data is not found in the LocalStore.
Additionally, this updates EdenServer to manage the BackingStore objects. It
maintains a map of the BackingStore objects created for each known repository.
Reviewed By: bolinfest
Differential Revision: D3409602
fbshipit-source-id: 2920dc4c24ee1ec37efb542f058d0d121ceb5532
Summary:
The _get_hg_dir() function had an incorrect return statement caused "eden init"
to fail with mercurial repositories, unless the path pointed to a shared
working directory.
Also update the code to store the path to the mercurial repository, rather than
the path to the .hg directory. Unlike git, mercurial requires the repository
path itself, and refuses to work if given a path to the .hg directory.
Reviewed By: bolinfest
Differential Revision: D3409808
fbshipit-source-id: 22ee7748422b614f5af57aff07fcfa517af846d9
Summary:
This revision introduces two complementary changes:
* `eden daemon` no longer runs in the foreground.
* There is now an `eden shutdown` command to kill the daemon.
When `shutdown` is called, it tells the Thrift server to shutdown.
In turn, this causes `EdenServer::runThriftServer()` to exit,
which causes `EdenServer::run()` to exit.
Reviewed By: simpkins
Differential Revision: D3402347
fbshipit-source-id: 80032ba53eb69b3f69bef9d7cd169f93500c833c
Summary: Add stats tracking for eden. Thought it would be best to collect some feedback before I go ahead and add tracking for all of the operations in the "dispatcher_ops" structure.
Reviewed By: simpkins
Differential Revision: D3387902
fbshipit-source-id: 2fdb22d3b7d0248f67255614f82442379e53e212
Summary:
D3406773 included a change which compiles on clang and gcc-5.x, but fails to
build with gcc-4.9.
This looks like a bug in gcc-4.9's handling of list initialization. Overload
resolution for non-initializer-list constructors should be attempted if
no suitable initializer-list constructors are found, but gcc-4.9 does not
appear to do this.
Reviewed By: bolinfest
Differential Revision: D3410142
fbshipit-source-id: f34125000eb3fa949c2427aa4ffbd4ef92942cd7
Summary:
Update deserializeGitBlob() to work on an IOBuf, rather than an rvalue
reference to a string.
The ugliness about having to wrap a std::string in a managed IOBuf is now
hidden inside the StoreResult class, rather than being something that the
GitBlob code has to know about.
Reviewed By: bolinfest
Differential Revision: D3403977
fbshipit-source-id: 0c58c019557050d6e201c1a462fa051c2526674a
Summary:
Add a new StoreResult which wraps the std::string returned by RocksDB.
This replaces the std::unique<string> that LocalStore::get() used to return.
This lets us avoid a memory allocation. StoreResult can also represent a "not
found" result, so that this case can be processed efficiently without having to
throw an exception.
Additionally, StoreResult is move-only so we can't ever unintentionally copy
the string data, which is potentially expensive. It also provides APIs for
creating IOBuf wrappers, or moving the string to the heap so we can create an
managed IOBuf around it.
Reviewed By: bolinfest
Differential Revision: D3403958
fbshipit-source-id: ab0c304988a53eda50341ecc2f96ae5235e5260c
Summary:
Add a new ObjectStore class, which will eventually contain both a LocalStore
and a BackingStore. The LocalStore will be a cache of data loaded from the
authoritative BackingStore. The ObjectStore API will hide the work of querying
the BackingStore and updating the LocalStore when data is not already available
in the LocalStore.
For now ObjectStore only contains the LocalStore, but I will add BackingStore
functionality in subsequent diffs. This diff simply updates all call sites to
use the ObjectStore instead of directly accessing the LocalStore.
Reviewed By: bolinfest
Differential Revision: D3403898
fbshipit-source-id: 47b8c51a7717a4c7c29911a7085b382521a8c0db
Summary:
Previously Hash objects could not be assigned to after they were created, since
they contained a const member. This makes the data non-const, so a Hash
variable can be replaced to contain new contents after it is created.
This also adds a default constructor, which zero-initializes the hash. The
default constructor makes it possible to declare a Hash with a 0-value at one
location, and then set it to the desired value at some later point.
Reviewed By: bolinfest
Differential Revision: D3406773
fbshipit-source-id: 41e2c7e3ad5bc4d14813be4adaa03866701380f6
Summary:
- Add a Sha1Key class that can more efficiently compute the key for
file content SHA-1 values, without having to copy it into a new std::string
object. (In practice fbstring would have avoided having to actually allocate
memory, but it was still an extra data copy.)
- The code was always converting the hash keys to hex on get and put
operations, just in case it needed it if an error occurred. This diff
changes the code to only compute the hex value if an error actually occurred.
Reviewed By: bolinfest
Differential Revision: D3403889
fbshipit-source-id: 5abd8ef202cb00677a84a03a82e2a3d21f16cd2f
Summary:
Update several places to use std::array rather than plain C arrays, using
folly::make_array() to automatically deduce the correct type when necessary.
Reviewed By: wez
Differential Revision: D3370445
fbshipit-source-id: b7642cf3a9b08eac817988bf95679bf5e584ef72
Summary:
Add an HgImporter class for importing mercurial data into the eden local store.
At the moment this only includes code for importing revision manifest data as
tree objects, and does not yet include code for importing file blob data.
Reviewed By: bolinfest
Differential Revision: D3367958
fbshipit-source-id: 58049bf1594b3c27d676c5ebe778917b4043fccf
Summary:
Update deserializeGitTree() to accept an IOBuf object. IOBuf objects can
easily wrap other buffers, so this can still easily support ByteRange objects
as well.
Being able to use IOBuf's Cursor class ended up simplifying the logic a bit as
well.
Note that using IOBuf does require copying the name and mode data out of the
buffer when we read it (using the readTerminatedString() API). This is
necessary since the data may not be stored contiguously in the IOBuf. However,
this shouldn't impact performance much: we already need to copy the name data
into a std::string anyway. For the mode, most modern platforms can avoid doing
a heap allocation for this small string.
Reviewed By: bolinfest
Differential Revision: D3357255
fbshipit-source-id: 5b6e1bc93199849327409a8039266d7dc4f3afdf
Summary: Add a GitTreeSerializer class for serializing git tree data.
Reviewed By: bolinfest
Differential Revision: D3356770
fbshipit-source-id: d04bc9788117272504c2faa335b3648e4ac93e81
Summary:
Previous to this change, if `~/local/.eden` did not exist when `eden daemon` was
run, then it would fail.
Now the logic to create `~/local/.eden` is encapsulated in `_ensure_dot_eden_folder_exists()`.
We do not call this for all subcommands because we want to make sure that running
`eden --help` does not have the side-effect of writing `~/local/.eden` if it does not exist.
Reviewed By: wez
Differential Revision: D3397057
fbshipit-source-id: a3f974f367058d9e4ebd515c78423e54edc179cc
Summary:
`eden health` will return with exit code 0 if both of the following are true:
* The Thrift client is up and running.
* The status of the client is either ALIVE or STARTING.
Reviewed By: wez
Differential Revision: D3395582
fbshipit-source-id: ba668d26acae73a51fbae8aca2b4979156c0c50f
Summary:
When running the CLI "daemon" command, we try killing the underlying eden
process group when we receive SIGTERM or SIGINT. (We really only want to kill
the main eden process, but we currently have to kill the entire process group
due to how sudo works.)
Since the privhelper process runs as root and is part of this process group, we
can get an EPERM error back. This was causing the CLI to fail with an
unhandled exception backtrace. This diff updates the code to ignore EPERM.
Reviewed By: bolinfest, wez
Differential Revision: D3384121
fbshipit-source-id: 39b2364d8c921b1d84a8902566fe9af2a370e4e5
Summary:
Define a constant for the path directory separator ('/'). We currently plan to
normalize all pathnames to use forward slash as a directory separator, even on
Windows. This will simplify the bulk of the eden code.
Even though we don't plan to make this separator character configurable, it
still seems useful to define a symbolic constant for it.
Reviewed By: bolinfest
Differential Revision: D3377110
fbshipit-source-id: cf6b158824e0cb318cc94838618506e78d868c9f
Summary: Add functions for checking if one path is in a subdirectory of another path.
Reviewed By: bolinfest
Differential Revision: D3367035
fbshipit-source-id: 3bd10d88bded2ccf0066d273bc5fe5c447287cb6
Summary:
The reverse path iterators derived from the forward iterators, and overrode the
pre-increment operator. Unfortunately they didn't override post-increment,
which meant that post increment actually advanced them forwards instead of
backwards. (The base class's post-increment calls the base class's
pre-increment, since this method is not virtual.) The reverse iterators also
weren't overriding the decrement operators added in D3366877, so decremening a
reverse iterator would move it in the wrong direction.
This fixes all of those issues by making the ComposedPathIterator template
smarter so it can correctly handle both forward and reverse iteration.
It also now contains the logic that the "begin" iteration for absolute paths
needs to start one character in. ComposedPathIterator needs to be aware of
this in order to stop at the correct character when iterating backwards.
Reviewed By: bolinfest
Differential Revision: D3376727
fbshipit-source-id: c61c86c1a9233507f8297015439e416680aaa39b
Summary:
This modifies the iterator behavior to so the behavior is a bit cleaner
with respect to empty paths. It is valid to have an empty relative path,
and there are legitimate use cases where this is useful. For instance,
calling dirname() on a RelativePath with a single component will result in
an empty path. It is useful to use this empty path to refer to the parent
directory, to which the path is relative. Therefore it is also useful to
be able to include the empty path when iterating through the parent
directories of a path.
This removes RelativePath::begin() and RelativePath::end(), and replaces
them with a RelativePath::paths() function. paths() returns a struct with
a begin() and end() function, so it can be used in range-based for loops,
and has the same behavior that begin()/end() did. This also adds a
RelativePath::allPaths() function, which also includes the empty relative
path in the results.
Reviewed By: bolinfest
Differential Revision: D3366877
fbshipit-source-id: 3d92b600f07b993925f88d4f1e619b6c1705fb82
Summary:
Unfortunately operator->() can't really be implemented properly, since
ComposedPathIterator doesn't actually point to an already-existing path piece.
This removes the broken implementation, and leaves a comment in its place.
Reviewed By: wez
Differential Revision: D3366836
fbshipit-source-id: acbd0332a0a7a1fc9eac114ef0c06ed605adbbf3
Summary:
PrivHelper serializes messages and sends it over to PrivHelperServer who verifies that mount point exists, cleans up bind mounts for the FUSE mount, and undoes FUSE mount.
Some repeated code in this diff since I was unsure on the protocol for that - let me know if/where I should generalize functions to avoid this.
Reviewed By: simpkins
Differential Revision: D3361955
fbshipit-source-id: a7324fb9660912d6c2b753e15b1fa6061c0d5261
Summary:
This avoids translation from string->Hash in the common case
where the file is unmodified and its hash is read directly from
the store rather than computed from the overlay.
I'm guessing I should use `unique_ptr` as the return value throughout?
Reviewed By: simpkins
Differential Revision: D3355773
fbshipit-source-id: 50dff879a78b3d6ff49f86b856866ca28808c4f7
Summary:
Other tools, such as Buck, will benefit from being able to get
the SHA-1 of a file without having to read the entire contents
of the file (or do the associated computation that is proportional
to the size of the contents of the file).
Reviewed By: simpkins
Differential Revision: D3345828
fbshipit-source-id: 360bb268793369af75f408208e8211d8b9db146d
Summary: Updated python CLI to include subparser for unmount command and added wrapper functions that hand over execution to privhelper process. Unmount currently requires client_name at the command line.
Reviewed By: simpkins
Differential Revision: D3359517
fbshipit-source-id: ff05e90bcdb96ecad63f37634c69dbeef429c90f
Summary:
Change Hash::getBytes() to return a folly::ByteRange rather than a
std::array<uint8_t, 20>. This makes Hash more convienent to use with existing
APIs that accept a ByteRange. (For instance, IOBuf.)
There were only 2 call sites using the existing getBytes() functionality,
and they only used the data() method on the returned std::array, so they don't
have to be updated at all to use a ByteRange.
Reviewed By: bolinfest
Differential Revision: D3354581
fbshipit-source-id: 8f2a3c196e59620fb5b0fb2caf4d1d7f26e1d2c4
Summary:
This is an alternative to `InodeDispatcher::lookup()` that does not result
in `RequestData::get()` being called, which is important for requests that
do not originate from FUSE.
The `::lookup()` call takes the `InodeBase` and invokes its `getattr()` method,
which in the case of `TreeEntryFileInode::getattr()`, creates a `fusell::Dispatcher::Attr`
whose constructor calls the following:
```
Dispatcher::Attr::Attr() : timeout(1.0) {
memset(&st, 0, sizeof(st));
auto& req = RequestData::get();
auto mount = req.getChannel().getMountPoint();
st.st_uid = mount->getUid();
st.st_gid = mount->getGid();
}
```
So the only reason this is done at all is to set `st_uid` and `st_gid`.
And then the only reason `Attr` is needed is to set the following fields on a `fuse_entry_param`:
```
attr
attr_timeout
entry_timeout
```
So it is possible that we can find a better way to streamline all of this, but this was
the easiest shortcut I could implement myself right now.
Reviewed By: wez
Differential Revision: D3351819
fbshipit-source-id: b095c085ee4a9b9a81438db093869fd0acf1f8ad
Summary:
I am going to submit some changes to `InodeDispatcher.cpp` in a
follow-up diff where I definitely want to benefit from auto-format,
so to make that diff less noisy, I'm doing the auto-format in this revision.
Reviewed By: wez
Differential Revision: D3351888
fbshipit-source-id: f8fdebe43f247e2522f1f14b3ee4bac37f564c70
Summary:
Add a function to compute the sha1 content hash for an overlay file.
We persist the computed hash in an extended attribute in the underlying overlay
file so that a subsequent read of the attribute doesn't require opening the
file to recompute it.
Each time the file is mutated, we blow the cached status of the file.
Each time the sha1 attribute is read, if the cache is blown, the content
hash will be recomputed and set in the overlay file.
Each time the file is flushed or sync'd, if the cache is blown, the content
hash will be computed and set in the overlay file.
Reviewed By: bolinfest
Differential Revision: D3302412
fbshipit-source-id: bd45c7a24b732bd0b7474b7f96e82936870b2117
Summary:
This is part 1 of 2 diffs. This one adds some plumbing to make it possible
to read the xattr attribute from an overlay.
It doesn't do anything to ensure that it is set; the next diff in this series will take care of that.
Reviewed By: bolinfest
Differential Revision: D3302410
fbshipit-source-id: 47406a9c75f29743691d396676c691bcb99c4760
Summary: This logic should be shared by the Eden CLI as well as unit tests.
Reviewed By: simpkins
Differential Revision: D3348300
fbshipit-source-id: c87b1f03f16560323f3d7685063bb6466c39efe2
Summary:
We look this up via the mount point or eden mount object instead.
I've also removed the mercurial library stuff that was added to support the now defunct lamehg fuse we had in the earlier days.
simpkins' new importer doesn't use these and it resolves our CI mismatch issue.
Reviewed By: bolinfest
Differential Revision: D3349698
fbshipit-source-id: 5f4ec16b76042959cd1e3184f46bb3526fbaf74c
Summary:
Update the CLI to support running "init" with a mercurial repository.
This is just some bare bones framework code at the moment. It doesn't actually
import any data from mercurial at the moment, and mounting doesn't work.
Reviewed By: bolinfest, wez
Differential Revision: D3345426
fbshipit-source-id: 72c31ac8d2aac2a16e0a7d6f0425eb4ca218d487
Summary:
Use the hex-encoded version of the key in the RocksException if _get() fails,
rather than the raw binary data.
Reviewed By: bolinfest
Differential Revision: D3345355
fbshipit-source-id: cd8dc644a56ca3d5f3b9a9a0f5cc789b142f0bda
Summary:
Buck is [currently] built with Java 7, so it can only use third-party dependencies
that are also Java 7.
Reviewed By: simpkins
Differential Revision: D3342367
fbshipit-source-id: 4370fd152e7d2055495e783de68a6bb59867bee5
Summary:
This adds a new API to `PrivHelper`: `privilegedBindMount()`.
Similar to `privilegedFuseMount()`, this sends a message to the privileged helper,
which is running as `root`, so it can set up the specified bind mount.
The changes in the `privhelper` directory parrot what was done to support `privilegedFuseMount()`.
Now, once the primary mount for a client is created, any bind mounts listed in the
config for the client are set up. This logic is introduced in `EdenServer.cpp`.
Reviewed By: simpkins
Differential Revision: D3296660
fbshipit-source-id: 61296f35e5c3a6f232a1c17e0f296dd5d3b5ec06
Summary:
`glob()` prevents us from adding `.cpp` files to this directory that do not get compiled.
The header without the long path makes it clear that it is in the same directory as `PrivHelperTest.cpp`.
Reviewed By: simpkins
Differential Revision: D3301264
fbshipit-source-id: f58351a4f2857c8911f0bf82ae67136920fcb998
Summary:
Add a new class to serve as a single location where we can store all
information about a single eden mount point. Currently this contains the
MountPoint, LocalStore, and Overlay objects. This allows the TreeInode class
to just store a single pointer to the EdenMount, rather than having to track
these three objects separately.
In the future we could consider also keeping a copy of the ClientConfig in the
EdenMount object, but I haven't done that for now.
Reviewed By: bolinfest
Differential Revision: D3321355
fbshipit-source-id: 8a39bb49822ca8e90c88b2a834b59230d2f91435
Summary:
Enables mkdir in the overlay area.
I had to add some `lstat` calls in to the overlay dir reader because we depend
on knowing at least whether a node is a dir or not at the next level up.
When I run the test suite, the mounts are on my `/tmp` filesystem. When I run
eden manually, they are on my `/data` filesystem. The latter (xfs) does not
populate the type bits. This meant that the test suite passed but manual
testing did not.
Adding the `lstat` calls is a little unfortunate. On OS X there is a bulk
operation that combines `readdir` and `lstat` so that there are fewer syscalls.
We don't have an equivalent for Linux.
Reviewed By: bolinfest
Differential Revision: D3301532
fbshipit-source-id: e228f4a392f90aa491fec62e8b98471a8acecff2
Summary:
We still have naked pointers at the handoff to the kernel, but now
have a cleaner implementation at the level that we're going to be working at
day to day.
I also renamed `FileHandle::release` to `FileHandle::releasefile` so that it
isn't visually ambiguous with `std::unique_ptr::release` in the
`Dispatcher.cpp` code: `fh.release()` vs `fh->release()` look similar but are
dramatically different in behavior.
Reviewed By: bolinfest
Differential Revision: D3309455
fbshipit-source-id: f8cf055bcd51121048a20f0202988cf0aef1f085
Summary: Now that we have integrated this functionality, we don't need these
Reviewed By: simpkins
Differential Revision: D3307034
fbshipit-source-id: 590469e9e7dec9c6b7d4f7b4e9a8d8c40d6ff552
Summary:
`creat(2)` and `open(2)` could decide to create a PassThru file handle.
This diff removes that usage. There is a TODO here around handling `O_EXCL` properly.
I'm punting this to a follow-up diff.
Reviewed By: bolinfest
Differential Revision: D3301387
fbshipit-source-id: d35104c536396e7fd064d786f3d5592ecfcbfecf
Summary:
Centralize and delegate most (all?) of the content sensitive portions of file accesses into the FileData class.
Add tests to show that we can write to the overlay file and that the stat data is consistent with the result.
Reviewed By: bolinfest
Differential Revision: D3301251
fbshipit-source-id: a09316ad61c6ef4c656bc5d6dbd43f906abb7932
Summary:
This is the workhorse for adjusting the state that we track for the file data.
It handles both overlay and Tree backed data cases and moving from the latter
to the former depending on the open flags provided when a file handle is
opened.
This diff handles more cases than we have tests for. Those will be covered later in this stack of diffs.
Reviewed By: bolinfest
Differential Revision: D3301213
fbshipit-source-id: c1dab40c0ad205ce6cee820043b70dd886e78431
Summary:
Another step towards making TreeEntryFileInode overlay aware, this diff implements:
- stat
- readlink (although it is not possible to create symlinks yet)
- open
Reviewed By: bolinfest
Differential Revision: D3255158
fbshipit-source-id: 3f90b624e629ef279d6cc32e1d82787ee24796eb
Summary:
This allows us to share the same state between multiple open files and
also helps to reduce the size of inode instances that are otherwise inactive;
when there are no outstanding references to the data, we can drop it and forget
it.
Reviewed By: bolinfest
Differential Revision: D3301198
fbshipit-source-id: f0b3fc73a666ec2033c7a22e9eb587d3212cf966
Summary:
Update the PrivHelperConn code to copy file descriptors in and out of the cmsg
data using memcpy(), rather than dereferencing reinterpret_cast'ed pointers.
These cast's break C/C++'s strict aliasing rules, and gcc complains about this.
Reviewed By: bolinfest, wez
Differential Revision: D3306762
fbshipit-source-id: d36fb5ee02a52da2b60e86fb4a0fdb4910fc72d7
Summary: This should set us up to have `eden mount` perform the bind mounts.
Reviewed By: simpkins
Differential Revision: D3296370
fbshipit-source-id: 5d8c21308074b357bad3ace72cec157adb5f8b56