Commit Graph

430 Commits

Author SHA1 Message Date
Adam Simpkins
ec85eb7ec7 fix warnings triggered with -Wshadow-compatible-local
Summary:
Make sure the eden coded compiles cleanly with -Wshadow-compatible-local

Pretty much all of the warnings were issues with lambdas shadowing names from
their parent context (even though they didn't ask to capture those names from
the parent).

Reviewed By: wez

Differential Revision: D4644849

fbshipit-source-id: 66629cd98b5af4760f3fbb256e44c0bc47e52316
2017-03-02 13:32:51 -08:00
Wez Furlong
c543f89404 add symlink support to the overlay
Summary:
Previously, we would only allow you to consume a symlink that was
checked into a revision, but not allow you to create a new one.

This diff adds support for making new symlinks.

It turns out that the code that we had for readlink had an unused code path for
reading an actual symlink out of the overlay.  My first pass at this diff tried
to make the materialization code generate such a symlink, but it was breakin
some other assumptions in the rest of the code.

This version of the code just stores the symlink target in the file contents.
This means that we can simplify readlink to just calling `readAll` on the
underlying file data, and that will load the contents out of the storage layer
if the file wasn't materialized.  This helps simplify things a bit more.

Reviewed By: bolinfest, simpkins

Differential Revision: D4604016

fbshipit-source-id: 3138d0f9880639d2bbeaf4bb03bef3f021c3ecb3
2017-03-02 08:18:45 -08:00
Wez Furlong
2fc9fc155c enable StreamingEdenService
Summary:
In order to send realtime subscription information to watchman
we need to use the thrift streaming data functionality.  Since this
isn't supported in java we originally broke this out into its own
thrift file.

It's now time to hook this up; this diff adds some plumbing so that
our thrift server knows to implement this interface.

This was made a bit more complicated because the inheritance in
the thrift definitions causes some ambiguities in the header maps
that buck uses to manage the includes.

I also had to rename the file from `eden.stream.thrift` to `streamingeden.thrift`
to avoid a problem with the period in the middle of the identifier.

For the server side of things, I've removed the `:thrift-cpp2` target
and replaced it with `thrift-streaming` instead.

This diff doesn't implement the endpoint, it is just setting things
up for the follow-on diff that does.

Reviewed By: bolinfest

Differential Revision: D4640639

fbshipit-source-id: b3625b0ff33a42945bf523944beed050f549a18c
2017-03-01 23:12:50 -08:00
Wez Furlong
82612ba4d6 remove getMaterializedEntries thrift API
Summary:
I originally added this to facilitate `hg status` but we didn't end up
using it.  Rather than keeping it around and updating the tests in the `.eden`
dir diffs that follow, let's just remove it.

Reviewed By: simpkins

Differential Revision: D4610768

fbshipit-source-id: 158c0207f88980e86aeeddf75e6fd49763d2a402
2017-03-01 08:19:29 -08:00
Adam Simpkins
09006d0e4a Drop AutoHeaders.RECURSIVE_GLOB from TARGETS files
Summary:
Remove AutoHeaders.RECURSIVE_GLOB from all of our TARGETS files.  This is a
Facebook-internal directive that was added as part of a codemod.  The vanilla
open source buck does not know about this directive, and can't parse TARGETS
files that contain it.

All of the affected rules are cpp_unittest() and cpp_binary() rules that don't
actually have any header files, so these parameters didn't actually have any
effect.

Reviewed By: andrewjcg

Differential Revision: D4633748

fbshipit-source-id: 6227fe9b7f2ea838a0c72408ca195685bcae9f5a
2017-02-28 21:46:39 -08:00
Michael Bolin
22994cb1e8 Make oss install script less hacky using --out argument to buck build.
Summary:
The `--out` option was not available when this script was originally written,
but this is exactly the sort of hacky shell-scripting it is desgined to
replace.

Reviewed By: mzlee

Differential Revision: D4625460

fbshipit-source-id: 9c884d0274bb5ebda8c59a7c9c6b2be22efd6265
2017-02-28 12:12:35 -08:00
Adam Simpkins
c764deb101 implement path component iteration
Summary:
Add components() and rcomponents() methods to ComposedPathBase.
These functions return a pair of iterators allowing you to iterate through the
components in the path.

This is similar to the existing paths() and rpaths() method which iterate
through path prefixes.

Reviewed By: wez

Differential Revision: D4609907

fbshipit-source-id: 698e5d58db897bbc760f2a2de90ef430fcb03a52
2017-02-23 20:22:21 -08:00
Adam Simpkins
0515c8c6a4 fix a bug incrementing the wrong index during TreeInode checkout
Summary:
In TreeInode::computeCheckoutActions() the code incremented the wrong index
when we reached the end of the new tree before the old tree.

panda-facepalm

Reviewed By: bolinfest

Differential Revision: D4609906

fbshipit-source-id: 61ca6baabe4eebb033c48b742562558466a45104
2017-02-23 18:29:41 -08:00
Adam Simpkins
b766aba11c invalidate the FUSE cache properly during checkout
Summary:
Invalidate directory entries as they are modified during the checkout
operation.  We never need to invalidate inode contents, since we always update
files by replacing the file with a new inode.

Reviewed By: wez

Differential Revision: D4577942

fbshipit-source-id: afde3040960b076d2cd384301a89cd7bdfce5a6a
2017-02-22 18:37:42 -08:00
Adam Simpkins
645b6fca0a update "eden daemon" to also pass through KRB5CCNAME
Summary:
When the "eden daemon" CLI command has to run edenfs via sudo, also explicitly
pass through the KRB5CCNAME environment variable.  This will allow edenfs and
it's hg import helper to find the correct Kerberos credentials if it needs to
connect to a remote server.

Reviewed By: wez, bolinfest

Differential Revision: D4584132

fbshipit-source-id: 6c72e4eca76d81b3d4b53a9e72a0e8388da30a7d
2017-02-22 18:37:42 -08:00
Adam Simpkins
82d8d57ea7 handle forced checkout of newly added files with conflicts
Summary:
This updates CheckoutAction to make the old TreeEntry object optional.
This is necessary to support situations where the entry did not exist in the
old source control tree, but does exist in the new tree and also exists locally
on the file system.  This situation is always a conflict, so it only applies
when a force checkout is being done.

This diff also refactors the TreeInode::computeCheckoutActions() logic to
combine the processNewEntry/processRemovedEntry/processEntry lambdas into a
single function, as suggested in the code review for D4538516.

Reviewed By: wez

Differential Revision: D4577924

fbshipit-source-id: 9435205b541a12c75a8b9bd2b2b599a339a2f6c8
2017-02-22 12:21:48 -08:00
Adam Simpkins
84bd24cf96 add a getCurrentSnapshot() thrift call
Summary:
Add a thrift call to get the ID of the snapshot currently checked out in a
given eden mount.

Reviewed By: bolinfest

Differential Revision: D4570729

fbshipit-source-id: 208ceb309aacfb07e30ddb8fdfc0950a3d7d748b
2017-02-22 12:21:48 -08:00
Adam Simpkins
186867b8d2 remove the remaining calls to getPathBuggy()
Summary:
Fix the FileHandle code that was calling getPathBuggy() when writing journal
entries.  This fixes a crash when writing to an unlinked file.

Also remove an unused call to getPathBuggy() in TreeInodeDirHandle.

These were the last places calling getPathBuggy(), so I have now deleted this
method entirely.

Reviewed By: bolinfest

Differential Revision: D4570606

fbshipit-source-id: a1c4aab10df96e2aeee78fca6d5808db28ad0cbf
2017-02-17 18:44:32 -08:00
Adam Simpkins
fb20f999cd forcibly unmount eden mounts during shutdown
Summary:
When the privhelper process detects that the connection to the main eden
process has closed, have it forcibly unmount the eden mount points.
(This does the equivalent of "umount -l")

This is needed to properly clean up mount points which contain bind mounts
inside them.  If eden has already exited we will not be able to unmount the
bind mounts, and therefore we won't be able to unmount the parent eden mount
directory either without using MNT_DETACH.

In the long run we will probably want to update the eden process to shut down
gracefully on SIGINT or SIGTERM.  This would let us unmount everything cleanly
if we did the unmount operations before we stop responding to FUSE requests.
However, the privhelper still needs to be able to clean things up properly if
eden crashes, so this lazy unmount behavior will still be desirable in the long
run.

Reviewed By: wez

Differential Revision: D4548030

fbshipit-source-id: b60189181bccc6f126998c92fd9a2e47593c96b3
2017-02-16 21:38:36 -08:00
Adam Simpkins
17a7f05c55 improve detection of bind unmount status
Summary:
In the privhelper code we want to wait for the bind unmount to really complete
before proceeding.  Previously this was done by repeatedly calling statfs()
until it fails.  However, on my system the statfs() call continues to succeed
even after the unmount operation, causing bindUnmount() to never think that the
unmount has finished.  (I'm currently running Linux 4.6.7)

This updates the code to check the file system ID before and after the unmount,
and also stop once we see the filesystem ID change.  I also replaced the
Linux-specific statfs() call with the POSIX-standard statvfs() call.

Reviewed By: wez

Differential Revision: D4547938

fbshipit-source-id: 51f53c0c901331de80fb908ae30033a0b56091b4
2017-02-16 21:19:52 -08:00
Adam Simpkins
0687431924 implement EdenMount::checkout()
Summary:
This is the initial code for implementing checkout.

This isn't quite 100% implemented yet, but I think it's worth checking in this
code as-is, and getting the remaining functionality done in separate diffs.
In particular, a few operations aren't implemented:
- Removing a directory that was deleted in the new revision
- Replacing a directory that was replaced with a file or symlink in the new
  revision
- When doing a forced update, replacing a file or directory that did not exist
  in the old revision, but that was created locally in the working directory,
  and also exists in the new revision.

Reviewed By: wez

Differential Revision: D4538516

fbshipit-source-id: 5bb4889b02f23ab2048fcae2c8b7614340181aa6
2017-02-15 20:33:31 -08:00
Adam Simpkins
230273cdf8 ignore stale privhelper response messages
Summary:
If the eden process times out waiting on a response from the privhelper, the
next time it tries to send a request the old response may be available now.
In this case it will read the old response when expecting to read the response
for its new request.

This diff updates the code to discard the old response in this case and retry
reading another request.  This allows us to get back in sync with the
privhelper after a timed-out request.  Previously the code would simply give up
after finding a mismatching transaction ID in the response, causing it to
always be one response off, and preventing any future requests from succeeding.

This is still kind of hacky, and could definitely be improved in the future,
but for now it is better than always failing all future privhelper requests.

Reviewed By: wez

Differential Revision: D4547769

fbshipit-source-id: 396b57eccb28d938998e29594d336f35ba4fb1fb
2017-02-15 19:39:14 -08:00
Adam Simpkins
e37b39b8ef fix compilation issues with path comparisons
Summary:
Update PathFuncs so that only PathComponent defines comparison functions
between PathComponent and PathComponentPiece.  Previously PathComponentPiece
also defined methods for performing comparisions between PathComponent and
PathComponentPiece, resulting in ambiguous overload errors when you tried to
use them.  (The same also applied to the RelativePath and AbsolutePath types.)

Reviewed By: wez

Differential Revision: D4546326

fbshipit-source-id: 158f72138e6d81d02beaeb925bfcc59db673c51a
2017-02-15 17:59:58 -08:00
Adam Simpkins
ff7b05831f don't hang the privhelper forever waiting for bind unmounts
Summary:
Update PrivHelperServer::bindUnmount() to give up after 2 seconds instead of
possibly looping forever.

Also fix the unmount code in the eden process to rethrow the original exception
as-is.  Previously it was re-throwing it as a std::exception, losing the
exception details.

I think there are still other bugs here in this unmount code--it repeatedly
hangs for me on unmount.  (I believe it is trying to unmount things that aren't
actually bind mounts.)  However, I'll look into fixing those in a separate
diff.

Reviewed By: wez

Differential Revision: D4547711

fbshipit-source-id: af6bce0262e8314361450874ef998ebf7ad590e3
2017-02-15 16:56:29 -08:00
Adam Simpkins
fed342da30 store overlay files using inode numbers instead of paths
Summary:
Refactor the Overlay code to store data using inode numbers rather than the
affected file's path in the repository.  This simplifies the TreeInode code a
bit, as we no longer have to rename overlay files to stay in sync with the file
paths.  This also eliminates some crashes when trying to update overlay files
for inodes that have been unlinked (and hence no longer have a path).  This
also includes a few fixes to avoid writing journal entries for unlinked files
too.  Additionally this contains a few fixes to how mode bits are stored in the
overlay, and fixes a bug where create() was ignoring the mode argument.

Reviewed By: wez

Differential Revision: D4517578

fbshipit-source-id: c1e31497dcf62c322b0deff72b0a02675b0509ab
2017-02-10 14:17:52 -08:00
Adam Simpkins
6920577f4a remove TreeInode::getInode()
Summary:
InodeBase::getNodeId() already returns the inode number, so there is no need
for the TreeInode subclass to provide an alternatively-named method that does
the same thing.

Reviewed By: bolinfest

Differential Revision: D4500215

fbshipit-source-id: fbc595cf9e8f43efe44ed352a97e3a4974b5aed0
2017-02-10 13:36:52 -08:00
Adam Simpkins
4d60f09c1c speed up importing manifest data from mercurial
Summary:
It turns out that the manifest class provides an iterentries() method to
iterate through the (path, hash, flags) values.  Update the code to use this
rather than iterating through the paths and then having to look up the hash and
flags using the path.

This dramatically speeds up importing a commit with 1M files: the import time
drops from around 8 seconds to under 1.5 seconds.  (Of the 1.5 seconds, about
half is spent reading the manifest data, and half is spent formatting and
writing the chunks.)

Reviewed By: wez

Differential Revision: D4512538

fbshipit-source-id: ccfff162082d327d525224b2d7dc87f67bda5e22
2017-02-06 12:28:05 -08:00
Adam Simpkins
8b4c984b28 always load affected inodes in rename(), and update lock ordering
Summary:
This updates the TreeInode::rename() code to handle concurrency better.
In particular:

- The code now ensures that both the source inode being renamed and destination
  inode (if it exists) are loaded.  This simplifies issues when an inode is
  being loaded at the same time a rename is in progress.  This ensures that any
  pending load is processed before the rename takes place.  (All promises for
  the load might not be fulfilled before the rename completes, but the relevant
  TreeInode and InodeMap data structures are updated before the rename occurs.)

  This does mean that the rename code potentially might have to retry several
  times if the inode it began loading is no longer the affected source or
  destination or child once the load completes.  However, this seems like a
  reasonable trade-off, compared to dealing with the complications that would
  arise with the load code having to handle renames occuring before load
  completion.

- The code now implements proper lock ordering, to avoid acquiring locks in
  conflicting orders that might cause deadlock with other threads also trying
  to acquire the same locks.  The InodeLocks.md document has been updated to
  clarify the TreeInode lock ordering requirements.

Reviewed By: wez

Differential Revision: D4493526

fbshipit-source-id: 627393fafad90eb551aea62be7762d59ed043abe
2017-02-03 18:35:03 -08:00
Adam Simpkins
8884b46b3f move integration tests to eden/integration
Summary:
Move the integration tests from eden/fs/integration up one directory, to
eden/integration.

The main benefit is that this makes it easy to run just the edenfs unit tests
by running "buck test eden/fs/...".  These unit tests complete much more
quickly than the full set of integration tests, providing a faster test suite
to re-run repeatedly during development.  The integration tests can be run with
"buck test eden/integration/...", and the full set of tests can still be run
with "buck test eden/..."

Reviewed By: wez

Differential Revision: D4490247

fbshipit-source-id: 5ceb5a19526f56e1cb926f352fa30ad2f1212c05
2017-01-31 14:41:14 -08:00
Adam Simpkins
19a1d3c6b6 make sure FileData always open the overlay file for materialized files
Summary:
Most of the FileData logic assumes that file_ is open if the file is
materialized.  Fix the FileData() constructor to actually open file_ if the
entry is marked materialized, so that this assumption is actually true.

Reviewed By: wez

Differential Revision: D4471365

fbshipit-source-id: b399e6e5e25eaf12f4204f5835db01baa5a6cd78
2017-01-31 13:33:57 -08:00
Adam Simpkins
2c16e33026 do not hold tree locks recursively in getMaterializedEntriesRecursive()
Summary:
Update the code to release the parent TreeInode's contents lock when processing
its children.  This makes sure we do not hold the root lock for the entire
duration of this function.

Reviewed By: wez

Differential Revision: D4471289

fbshipit-source-id: dc4b50bd06fcceb7a7df3d7dd04cfc0d41285556
2017-01-27 16:47:57 -08:00
Wez Furlong
932ef52a55 implement glob thrift method
Summary:
This is to facilitate the watchman integration and draws on the
watchman glob implementation; the approach is to split the glob strings into
path components and evaluate the components step by step as the tree is walked.
Components that do not include any glob special characters can be handled as
a direct lookup from the directory contents (O(1) rather than O(num-entries)).

The glob method returns a set of filenames that match a list of
of glob patterns.

Recursive globs are supported.  It is worth noting that a glob like "**/*" will
return a list of every entry in the filesystem.  This is potentially expensive
and should be avoided.  simpkins is in favor of disallowing this as a forcing
function to encourage tool-makers to adopt patterns that don't rely on a
complete listing of the filesystem.

For now I'd like to get this in without such a restriction; it's also worth
noting that running `find .` in the root of the mount point has a similar
effect and we can't prevent that from happening, so the effect of the overly
broad glob is something that we need to be able to withstand in any case.

Unrestricted recursive globs will make it easier to connect certain watchman
queries in the interim, until we have a more expressive thrift API for walking
and filtering the list of files.

Note: I've removed the wildmatch flags that I'd put in the API when I stubbed
it out originally.  Since this is built on top of our GlobMatcher code and that
doesn't have those flags, I thought it would be simplest to just remove them.
If we find that we need them, we can figure out how to add them later.

Also Note: the evaluation of the glob is parallel-ready but currently limited
to 1 at a time by constraining the folly::window call to 1.  We could make this
larger but would need a more intelligent constraint.  For example, a recursive
glob could initiate N concurrent futures per level where N is the number of
sub-dirs at a given level.  Using a custom Executor for these futures may be a
better option to set an upper bound on the number of concurrent jobs allowed
for a given glob call.

Depends on D4361197

Reviewed By: simpkins

Differential Revision: D4371934

fbshipit-source-id: 444735600bc16d2c2185f2277ddc5b51f672600a
2017-01-26 12:47:05 -08:00
Adam Simpkins
4a6aaa5de4 load all materialized inodes when starting a mount point
Summary:
As part of the mount process, make sure we load InodeBase objects for all files
in the mount that are already materialized.

Other parts of the code assume that InodeBase objects are always loaded for
materialized files.  We never unload materialized inodes, but if the mount
point was unmounted then remounted we previously did not ensure to load the
materialized InodeBase objects.  This diff makes sure we load all materialized
inodes before starting the mount.

Reviewed By: bolinfest

Differential Revision: D4461193

fbshipit-source-id: 70d06fd01e2df333ce2816d5d7a392b0a5d6e1e6
2017-01-25 16:56:12 -08:00
Adam Simpkins
f8042c24d4 clean up some subtle issues with inode loading completion
Summary:
Fix two issues with inode load completion:

1) Make sure we release the parent's TreeInode lock before fulfilling promises
waiting on the inode load to complete.  Otherwise this can result in deadlock
if the promises have callbacks that then try to acquire the TreeInode lock.

2) Use unique_ptr<InodeBase> in the load completion code before we have
inserted the Inode into the InodeMap, rather than using InodePtr.  The new
InodePtr class relies on the InodeMap to handle ownership, and it cannot
destroy Inode objects unless they are in the InodeMap.  Using a unique_ptr
instead ensures that the InodeBase will be destroyed properly if an error
occurs between when we construct the InodeBase and when we insert it into the
map.

Reviewed By: bolinfest

Differential Revision: D4427889

fbshipit-source-id: 2627a4956735f4acebe5c259abd8de0430ff6177
2017-01-25 16:56:12 -08:00
Adam Simpkins
d1556c3601 implement Future-based recursive Inode lookup
Summary:
Rename the current EdenMount::getInodeBase() function to
EdenMount::getInodeBaseBlocking() and add a new getInodeBase() function that
performs the lookup asynchronously and returns a Future<InodePtr>.

This is implemented with a TreeInode::getChildRecursive() function that allows
asynchronous recursive lookups at any point in the tree.

Reviewed By: bolinfest

Differential Revision: D4427855

fbshipit-source-id: aca55e681a48c848b085b7fc6a13efe6cf0a4e3a
2017-01-25 16:56:12 -08:00
Adam Simpkins
20aaa1e2b8 Add InodeBase::getParent()
Summary:
Add methods to InodeBase to get the parent of the current Inode, and remove the
existing TreeInode::getParent() function.  The old TreeInode parent_ member was
never updated properly after rename or unlink operations, so it could return
incorrect information.

Reviewed By: bolinfest

Differential Revision: D4427103

fbshipit-source-id: cb66a014f745b4af31033e0a3c5405e29791b869
2017-01-25 15:06:05 -08:00
Wez Furlong
1e0397db50 raise the attribute TTL to maximum possible, add basic invalidation
Summary:
Thesis: Since we don't yet have `hg update` implemented, these attributes can be cached forever.

When we implement `hg update` we will explicitly inform the kernel to invalidate anyway.

... and it turns out that the `scmRemove` thrift call is making changes that
aren't visible to the kernel already so we need to hook up some basic
invalidation.

This diff includes the bare minimum code to facilitate this; I've added a
helper method to the fusell::RequestData class to test whether we are in the
context of a FUSE request.

In the EdenDispatcher::unlink method we need to explicitly invalidate if we are
not being called via FUSE.

I have mixed feelings about putting this code in here.  Given the initial
mental model I had in the prototype, this is the right place to put it, but it
does mean that all tree mutations must have via the Dispatcher in order for
invalidations to be routed to the correct places.  Do we want to move away from
this?

It wouldn't be the end of the world to expand this diff a bit to add
similar/appropriate invalidation calls to all the mutation methods in our
dispatcher implementations while we figure out if we want to keep this.

Reviewed By: simpkins

Differential Revision: D4452961

fbshipit-source-id: 9471f145242fce0620c6872b74b02c56c8d78af1
2017-01-24 09:39:06 -08:00
Wez Furlong
fb957d58ec allow interpolating USER and HOME in the ClientConfig class
Summary:
This mirrors the capability added in D4444058.

I don't think the `getenv` calls are awesome, but also don't think that we're
likely to want to override these outside of the environment either, so it
doesn't warrant plumbing these all the way through the eden cli, edenfs command
line flags and the layers to get to instantiate this object.

Reviewed By: simpkins

Differential Revision: D4446320

fbshipit-source-id: d5661e4f3e8dee82617eb6edddbcb9da5f4296d2
2017-01-23 23:54:38 -08:00
Wez Furlong
4829dfe319 change systemConfigDir to etcEdenDir
Summary:
While testing with the fb-eden rpm installed, I hit some integration
test failures.  These were caused by the integration tests picking up the
default post-clone hook configuration.

This diff changes our existing `systemConfigDir` option (which defaults to
`/etc/eden/config.d`) to `etcEdenDir` (which defaults to `/etc/eden`) and
adjusts the code that consumed `systemConfigDir` to construct the effective
value by computing `etcEdenDir + "config.d"`.

Doing this allows us to also default the `repoHooks` path to be
`etcEdenDir + "hooks"` rather than just hard coding `/etc/eden/hooks`.

The result of this is that our integration tests will now pass when `fb-eden`
is installed, because they override the `etcEdenDir` option and isolate their
`edenfs` processes from the globally installed configuration.

Reviewed By: bolinfest

Differential Revision: D4446321

fbshipit-source-id: 524fdb2f386fdf16dce42dce7661d07e13c0f0e7
2017-01-23 23:54:38 -08:00
Wez Furlong
220a2d4e1e clarify logging around running post-clone when it doesn't exist
Summary:
It's been bothering me that we claim to run and complete running
the globally installed hooks when they aren't there, so tidy up the logging.

Reviewed By: bolinfest

Differential Revision: D4446318

fbshipit-source-id: acda223e1df2302602ad9967b1aa1b334b8477c7
2017-01-23 23:54:38 -08:00
Wez Furlong
2e4610d35d cut ClientConfig over to InterpolatedPropertyTree
Summary:
This adopts the new InterpolatedPropertyTree class as
the configuration storage.

It doesn't introduce any actual configuration of the interpolation
replacements; that will be in a follow-on diff.

Reviewed By: bolinfest

Differential Revision: D4444157

fbshipit-source-id: 18ead8e9074d23b1154e81f012f0c90efced1350
2017-01-23 23:54:38 -08:00
Wez Furlong
774bc5b032 fixup paths used in post-clone and LameThriftClient
Summary:
This makes it possible to deploy via packaging (we'll put the par file
in the same dir as the extension) and eliminates a recursive buck build call
that was slowing down our integration tests; we can now just look up the path
from the environment that we set up in the TARGETS file to locate the par files
in the integration tests.

Reviewed By: bolinfest

Differential Revision: D4444096

fbshipit-source-id: 2ad6b6aa783563dc54a1cfd347a509c9919bb17a
2017-01-23 23:54:38 -08:00
Wez Furlong
7117c836d1 add InterpolatedPropertyTree class
Summary:
I don't know that I'm happy with the name; it started out as
just a helper wrapper to apply interpolation, but grew a bit to better
encapsulate how we access the config from the eden server.

This is a utility class that I'd like to use in place of the raw boost
property tree in our ClientConfig code.

It encapsulates the underlying property tree and provides a couple of simple
methods; one for loading data from a config file and one for looking up
a configuration value.

The configuration value lookup will replace tokens of the form `${KEY}`
with the value of `KEY` from its internal replacements dictionary.

This mirrors the interpolation handling that we do in the equivalent python
code for the eden client.

Reviewed By: bolinfest

Differential Revision: D4444075

fbshipit-source-id: 5d46d63f87caad4f409fbb981aa83165fcd6596d
2017-01-23 23:54:38 -08:00
Wez Furlong
dff225b60e add limited interpolation to eden config parsing
Summary: This allows us to templatize centrally managed config files

Reviewed By: bolinfest

Differential Revision: D4444058

fbshipit-source-id: a44372084d32dcf0b27922490ab40a48478a720d
2017-01-23 23:54:38 -08:00
Wez Furlong
bafc809c36 refactor and centralize ConfigParser calls to _loadConfig
Summary:
I'm going to follow up with some changes to substitute
some core env vars; this is just some prep.

Reviewed By: bolinfest

Differential Revision: D4444030

fbshipit-source-id: ab5d0e39aba14cdba39bb4867ec24955665723c6
2017-01-23 23:54:38 -08:00
Wez Furlong
8b8d6d319f add eden config command
Summary:
`eden config` dumps all parsed config values.

`eden config --get SECTION.OPTION` prints the value of the corresponding key.

Reviewed By: bolinfest

Differential Revision: D4434084

fbshipit-source-id: 63d9f72aa8794371e89a6d9e527bacaab17540de
2017-01-23 23:54:38 -08:00
Wez Furlong
9ff3def69f pass down paths for eden, edenfs, post-clone via environment in tests
Summary:
This makes it easier to reason about where these things can be found,
and in particular, allows passing the path through to the post-clone hook
script.  Although post-clone accepts some command line configuration options,
our hooks scheme doesn't seem to allow setting that in the eden configuration
file.

Reviewed By: bolinfest

Differential Revision: D4443872

fbshipit-source-id: ed92dbac3b91b7f7dc86a39a85b465907f290b47
2017-01-23 23:54:37 -08:00
Andrew Gallagher
14fa8d476e buckification: switch some raw Buck rules to use the buck_ wrappers
Summary:
We implement some sanity checking in wrappers, preventing some unsafe
consequences of using raw Buck UI in fbcode.

Reviewed By: Gownta

Differential Revision: D4453147

fbshipit-source-id: 3069898069c7e89223b133224f7c87d1a6b5886a
2017-01-23 19:30:02 -08:00
Adam Simpkins
e31889d295 fix a variety of lint issues
Summary:
Fix issues flagged by running "arc lint" on all eden files.  There are still 6
warnings outstanding, but these are either false positives or advice that we
intentionally are ignoring for legitimate reasons.

Reviewed By: bolinfest

Differential Revision: D4446615

fbshipit-source-id: 992f3c146f99d63935f849aa775dd6d611a04acf
2017-01-23 18:03:09 -08:00
Wez Furlong
7e29c0eafe raise the default startup timeout in the integration tests
Summary:
10 seconds isn't long enough for the tests to pass consistently on my devserver (I have a windows VM running on here, so there is more background crap running).

Raising this up to 30 gave me a successful test run.

Reviewed By: bolinfest

Differential Revision: D4442300

fbshipit-source-id: 5c3ae1fc0d0acb21c5bb44e36dae88d5180749d1
2017-01-23 09:10:58 -08:00
Adam Simpkins
251da81f36 update all copyright statements to "2016-present"
Summary:
Update copyright statements to "2016-present".  This makes our updated lint
rules happy and complies with the recommended license header statement.

Reviewed By: wez, bolinfest

Differential Revision: D4433594

fbshipit-source-id: e9ecb1c1fc66e4ec49c1f046c6a98d425b13bc27
2017-01-20 22:03:02 -08:00
Wez Furlong
26f5b43a93 fix running "eden" with no arguments
Summary:
Previously we'd emit an AttributeError.  This diff
fixes things up so that we show the help when no arguments are provided.

Reviewed By: simpkins

Differential Revision: D4434060

fbshipit-source-id: 838b1cab47118d8517e4dc0c5fde17fc69752b13
2017-01-18 21:56:57 -08:00
Adam Simpkins
09e9aa0819 add a mountpoint-wide rename lock
Summary:
This lock protects all operations that change the location of existing inodes
(rename(), unlink(), and rmdir()).  This is needed to avoid race conditions
during rename operations.

I also included some FIXME comments about properly updating location
information for unloaded inodes.  These should be taken care of in a subsequent
diff.

Reviewed By: wez

Differential Revision: D4361197

fbshipit-source-id: 5aaa1b8250f196b23207be8f3c0fd7f2603e6ae8
2017-01-17 15:03:20 -08:00
Adam Simpkins
a3af576c5a parallelize getSHA1()
Summary:
Update the getSHA1() thrift handler to get the file SHA1 values in parallel.

The inode lookup itself still happens serially at the moment.  We need to
provide a future-based version of EdenMount::getFileInode() in the future, and
change all existing callers of it to use the Future-based version.

Reviewed By: wez

Differential Revision: D4361091

fbshipit-source-id: 1abbc16df8c3edf52959a82f16a7f59e5d6d038b
2017-01-17 15:03:20 -08:00
Adam Simpkins
a598e8c334 implement Inode unloading
Summary:
This updates the InodePtr and InodeBase code to actually implement Inode
unloading and destruction.

At the moment we keep Inode objects as long as possible, and only unload them
during shutdown, or when the last reference to an unlinked inode goes away.
However, it should be straightforward to add on-demand unloading in the future
for Inodes that have not been accessed in a while.  The
TreeInode::unloadChildrenNow() function provides a template for what this would
look like (it would simply need to be changed to check an access time when
doing on-demand unloading).

Reviewed By: wez

Differential Revision: D4360765

fbshipit-source-id: a46b355f0ac0c25f873a156e62af5184317de735
2017-01-17 15:03:20 -08:00