Summary:
I was working on a new test and I got an error that `directaccess` must be
enabled for `inhibit` to work.
Reviewed By: simpkins
Differential Revision: D5077133
fbshipit-source-id: cc5235c845e3f299f96e1c901ef4aea18ca57b76
Summary:
We're using `window` to constrain resources and maintain an upper
bound on the number of futures that we spawn. Ironically, for a largish number
of already completed futures, this results in a recursive chain of `spawn`
calls in the `window` implementation and results in a stack overflow (the kind
that crashes a program, not the kind that answers programming questions).
This diff removes the use of `window` and replaces it with `collect`.
An earlier iteration of this diff just made all of these calls serially,
blocking the main thread. This at least does things in parallel; both
can torture the system for pathological walks, but at least the parallel
one should complete faster.
Reviewed By: simpkins
Differential Revision: D4932774
fbshipit-source-id: 40d39a85029f38ff69a530070efb879a81950451
Summary:
I had to add simple implementations to various things in `edendirstate`
in order to be able to run `hg histedit`. There is still a lot more to do, but
at least this gives us a starting point to iterate and a test to demonstrate
the most simple functionality.
Reviewed By: wez
Differential Revision: D5049308
fbshipit-source-id: 34727f633c003cacae44108eb3ece06590098c7b
Summary:
Note that we must specify quite a few extensions to get behavior that is
representative of how Hg works at Facebook.
Reviewed By: DurhamG
Differential Revision: D5057478
fbshipit-source-id: ee774a9b8dcebe82e4b19cc52f9b0b5a53e6420c
Summary:
Recall that we override `$HOME` in integration tests, so this will not overwrite
your personal `~/.hgrc` when you run an integration test.
An upcoming integration test for `hg histedit` that I am working on requires
this value to be set.
Reviewed By: wez
Differential Revision: D5051112
fbshipit-source-id: 2fd8541aa6504640b08337fdc22160e243beaae3
Summary:
`HgExtensionTestBase.assert_status()` was added in D4814422, but it was only
applied to `update_test.py`. This change updates the docstring (it appears to
have been copy/pasted from a nearby method), and makes use of it in the other
integration tests.
Reviewed By: wez
Differential Revision: D5050775
fbshipit-source-id: bb70740b6f455a84e7a22c3286c8ddbe2462f816
Summary:
Previous to this change, when I would add `import pudb; pudb.set_trace()` to do
some debugging, two annoying things would happen:
- I would have to edit the `TARGETS` file to add `pudb` as a dependency and
then `buck build eden/integration/hg` again.
- When I hit a breakpoint using `pudb`, I would have to go through the welcome
screen, change the theme, etc., because my settings were not found.
Now that I figured out what the problem was, I added instructions to the
`TARGETS` file to help others fall into the pit of success.
Reviewed By: wez
Differential Revision: D5050725
fbshipit-source-id: 1896f9f52eb056b3295b2d8e896dabb5d990ba22
Summary:
It is an error to not provide mode argument when open is
called with O_CREAT.
Reviewed By: oebeling
Differential Revision: D5020517
fbshipit-source-id: 87665ceb16ebdff797f8bf3d33bea082191e3121
Summary:
On OSX mercurial libraries are not installed in the standard pythonpath. This
option lets me pass a pythonpath to the hg importer script only, without
polluting the pythonpath for other subprocess invocations (such as python 3
based hooks, which are not happy with python2 pythonpaths).
Reviewed By: bolinfest
Differential Revision: D4992222
fbshipit-source-id: aa8e734970344215cc8783e164fd119c3ee6aed6
Summary:
These files are defining flags, so they should import gflags.
This import being missing causes open source build to fail.
Reviewed By: bolinfest
Differential Revision: D4992246
fbshipit-source-id: 61eb35b43ae6f9ff88bd34a9ebc8db6d80ac9d08
Summary:
When we say we are building "all" of Eden, we generally want to ensure we have
built the pyremote service that the Hg extension needs, as well.
Reviewed By: wez
Differential Revision: D4982118
fbshipit-source-id: 7282266e82e662a48b5ced6da6b7daedab8b65ea
Summary:
Update Dirstate::remove() to directly call TreeInode::unlink() instead of going
through the EdenDispatcher. Internal code should be able to use TreeInode
directly without going through the FUSE Dispatcher. This prevents us from
having to look up the TreeInode object by inode number again.
This also fixes TreeInode::unlink() and TreeInode::rmdir() to perform FUSE
cache invalidation correctly. EdenDispatcher used to do this for unlink()
calls, but it was missing for rmdir() calls.
Reviewed By: wez
Differential Revision: D4968832
fbshipit-source-id: 51fda5de196392c640436ca6ad7d8333e6799db9
Summary:
This replaces the shouldFileBeDeletedByHgRemove() function with a much simpler
and safer check.
It turns out this code was more complicated than it needed to be: it looked up
the inode in question and confirmed that it still existed, even though it's
calling function had already previously looked up the inode object.
I believe this deletes the last place in the code that still assumes that an
InodeBase object must be loaded if the inode has previously been materialized.
This also removes one of the last remaining places that directly holds the
TreeInode contents_ lock outside of TreeInode.cpp.
Reviewed By: bolinfest
Differential Revision: D4968834
fbshipit-source-id: a37bc0d7889eb81cca5b045cbc82732a53ef53d4
Summary:
This updates the Dirstate::addAll() code to use EdenMount::diff() internally,
instead of getStatusForExistingDirectory().
This lets us delete getStatusForExistingDirectory() and several other helper
functions that it used. The logic in getStatusForExistingDirectory() was based
on the incorrect assumption that files can only be modified from their expected
source control state if they are materialized. This could result in incorrect
results in a variety of cases (particularly after renames, or "hg reset"
operations). The EdenMount::diff() code does not have this problem.
That said, this new addAll() code does still have some performance issues--it
currently does a full tree diff for each input path. We should ideally fix
this in the future to only diff the necessary subtree for each path. However,
in the short term this trade-off seems worth being able to delete all of this
older, buggy code. diff() should be cheap enough in most cases that this won't
be a major problem unless a large number of paths are given as input.
Reviewed By: bolinfest
Differential Revision: D4968835
fbshipit-source-id: 1834aa98a26dcaa0e1c06c7ac25c57944fa1b5f7
Summary:
Make sure we use pointers to const GitIgnoreStack objects when possible.
Also change the code to store the root GitIgnoreStack object for a diff()
operation in the DiffContext object, instead of having EdenMount::diff()
allocate it separately on the heap. This will make it easier to consolidate a
bit more logic in the DiffContext class in the future. In particular, I
suspect we will want a version of diff() that only works on one portion of the
tree. Putting more of the functionality in DiffContext will make it slightly
easier to share code between the full-mount diff and the subtree diff
functions. This also happens to save a memory allocation for now.
Reviewed By: wez
Differential Revision: D4968833
fbshipit-source-id: 1dc33b3d44cdf00e93b22d810c3a736d27c13638
Summary:
Update the eden thrift APIs to support multiple thrift parents, and update the
hg extension to use the new thrift APIs.
Reviewed By: bolinfest
Differential Revision: D4949139
fbshipit-source-id: 98a7bd17ccad7be6a429df34ecfaf3fe7ae0b39a
Summary:
This updates the ClientConfig and EdenMount code to support storing two parent
commits.
This changes the on-disk SNAPSHOT file contents add an 8-byte header that
includes a file identifier and a file format version number, followed by up to
two commit hashes. The code currently can read either the old or new format
from the SNAPSHOT file. We should be able to drop the code for reading the old
format fairly soon if we want, though.
This diff only updates the ClientConfig and EdenMount code, and does not yet
update the thrift APIs or the eden mercurial extension yet. I will update the
rest of the code in a subsequent diff.
Reviewed By: bolinfest, wez
Differential Revision: D4943917
fbshipit-source-id: cf456e67b845aa0cf8b45c822985cb932df107b4
Summary: Add some unit tests that check the behavior of setattr() and getattr().
Reviewed By: bolinfest, wez
Differential Revision: D4941215
fbshipit-source-id: 6691f51aea742d0159a16112ca291546a8adc928
Summary:
The error messages for various failure cases in EXPECT_THROW_RE() and
EXPECT_THROW_ERRNO() were missing a space after the statement being executed.
Reviewed By: bolinfest
Differential Revision: D4941175
fbshipit-source-id: e10b4bff7db027acc5cc7ad5c12af248d55c7cd8
Summary:
This fixes "hg commit" so that it correctly updates the in-memory snapshot.
This has been broken ever since I added the in-memory snapshot when
implementing checkout(). The existing scmMarkCommitted() method updated only
the Dirstate object and the on-disk SNAPSHOT file.
This diff fixes checkout() and resetCommit() to clear the Dirstate user
directives correctly, and then replaces calls to scmMarkCommitted() with
resetCommit().
Reviewed By: bolinfest
Differential Revision: D4935943
fbshipit-source-id: 5ffcfd5db99f30c730ede202c5e013afa682bac9
Summary:
This updates how we build and package the eden hg extension, and how we find it
during integration tests.
- Update the extension to always look relative to its current location to find
the other modules it depends on. This ensures that the integration tests
always find modules from the local repository, and do not use the modules
installed on the system.
- Add a buck rule to unpack the python archive at build time. This is needed
for integration tests to use the local version of the module.
- Ensure that we install a correct `hgext3rd/__init__.py` module in the eden
extension directory. This is required to correctly set up `hgext3rd` as a
namespace package. This also unfortunately needs to be a `.py` file, and not
just a .pyc file. (The pkgutil.expand_path() code looks specifically for
directories containing `__init__.py` files, and does not check for
`__init__.pyc`.)
- Update the extension to only try importing the native thrift modules if we
are running python 2.7.6 or greater. Python 2.7.6 is the first that supports
unicode arguments to `struct.pack()`, which thrift requires. Python 2.7.5 can
import the thrift modules, but throws errors when trying to run them.
Reviewed By: bolinfest
Differential Revision: D4935279
fbshipit-source-id: 9af81736124c55476a5eb5beba9474a4371a639b
Summary:
Update the gitignore code so that patterns ending in a trailing slash correctly
match only directories. Fortunately we have file type information available
during TreeInode::computeDiff(), so getting the correct file type information
does not add any extra overhead. The older ignore checking code in
Dirstate.cpp can also do a reasonable job of getting file type information.
That code should also be removed at some point in the future, and use the
TreeInode logic for computing ignore status.
Reviewed By: bolinfest
Differential Revision: D4901326
fbshipit-source-id: 1222c8142876c91e1b80ec937ec84c0c28737224
Summary:
The temporary Hash object doesn't live long enough for the copy into
the string, so make an explicit temporary to extend its lifetime.
Reviewed By: simpkins
Differential Revision: D4900457
fbshipit-source-id: 948d919b929285c30ee1f9f0111cba20b60d4dc6
Summary: We overlooked this in the recent move around of the cli code
Reviewed By: bolinfest
Differential Revision: D4894230
fbshipit-source-id: 5c3ae274a182e5cc41df47100959f8cfbcb59c2d
Summary:
When tree manifest data is available, we don't need to parse the full
manifest any more, which is great because we can simply and quickly load the
root manifest and be done with a checkout operation. We can then quickly load
the manifest entry for a given directory path on demand, making us closer to our
ideal of O(what-you-use).
This diff plumbs in the manifest code from mercurial so that we can reuse
the decoder implementation to get the manfiest data and translate it into
something that we can put into our LocalStore.
There's a little wrinkle when it comes to files; mercurial doesn't support
a means for getting the file contents based on just its hash, we have to
provide the filename and revision hash for that. We have existing code
to create proxy entries in the store to map from a proxy hash to a tuple
with the appropriate data needed to resolve the file entry.
We need to extend its use to the trees that we generate because we need
to be able to propagate that path down to a child tree when we generate
the tree entry for a file for the first time.
If a tree manifest is not available for a given revision, we fall back
to the existing implementation that parses the full flat manifest.
Note: the flat manifest import generates entries with different hashes
compared to the tree manifest hash version. While we could make the
serialization match up, there's little risk of this causing problems
for us in practice, so punting on that for now.
Reviewed By: simpkins
Differential Revision: D4873069
fbshipit-source-id: 865001996745700586f571ed67371aed1668a6a7
Summary:
This tweaks the main.cpp and EdenServer.cpp code so that we can avoid having
separate internal and open source builds of the eden/fs/service:server library.
Only one small function is different between the two builds. This updates the
code so that this function is only used from main.cpp, instead of being used
inside EdenServer.cpp.
Reviewed By: wez
Differential Revision: D4889721
fbshipit-source-id: 97abda92c36752516fb8a8e73b4cf50580db8d79
Summary:
This change makes it so that all of the C++ code related to the edenfs daemon
is now contained in the eden/fs subdirectory.
Reviewed By: bolinfest, wez
Differential Revision: D4889053
fbshipit-source-id: d0bd4774cc0bdb5d1d6b6f47d716ecae52391f37
Summary:
Move the code for the command-line tool up one directory, out of eden/fs.
This better separates the code so that eden/fs contains code for the edenfs
daemon, while eden/cli contains code for the command line tool.
Reviewed By: bolinfest
Differential Revision: D4888633
fbshipit-source-id: 5041e292c5353d05122eefe5db3257289e31239a
Summary:
Fix a subtle crash during checkout when handling newly added entries that
already exist in the working directory: CheckoutAction passed the entry name to
checkoutUpdateEntry() as a PathComponentPiece. However, this
PathComponentPiece could refer to the entry name owned by newScmEntry_, and it
also passed newScmEntry_ into checkoutUpdateEntry() as an rvalue reference.
As a result, if the string data was stored invalidated by the move the name
would no longer be valid when checkoutUpdateEntry() tried to use it.
This bug is triggered by doing an "hg update --clean", where a file added in
the destination commit already exists on disk, and has an entry name of 23
characters or less. (The 23 character limit is fbstring's upper bound on
small string optimizations, where it will store the string data inline in the
object, causing it to be invalidated on move.)
This also fixes a crash in a VLOG() statement when the verbose log level for
TreeInode.cpp was set to 4 or greater.
Reviewed By: bolinfest
Differential Revision: D4882544
fbshipit-source-id: 917ede6eeae2224aaa0724b8b30324f3c3a5c924
Summary:
This is a bit too magical -- it's not clear that the thing produces an Options object. If you do know that you can chain further option setters off this thing, it's nice, but otherwise, the first impression is "what just happened?".
So, let's have one good way for doing things.
Reviewed By: yfeldblum
Differential Revision: D4863947
fbshipit-source-id: 3dfe83cfc077d47f604f47dcb21149fbaa2d2243
Summary:
Update the hg extension to implement dirstate.rebuild(). This is necessary for
the `hg reset` command. This also now implements dirstate.setparents() for
cases when there is only one parent.
Reviewed By: wez
Differential Revision: D4823780
fbshipit-source-id: 802de006e03860995095dc3af17acb2eb05f4e8b
Summary:
Update the checkOutRevision() thrift API to accept the commit ID as a
BinaryHash rather than as a hexadecimal ASCII string, to make it more
consistent with our other thrift APIs.
Reviewed By: wez
Differential Revision: D4823779
fbshipit-source-id: 6226f926533ef86d60f8b1d659bc5d912e2855e6
Summary:
The intent is to provide a way to locate the SNAPSHOT file
for tools that want to have a very fast way to figure out the commit
id without making any RPCs or subprocess invocations.
Reviewed By: simpkins
Differential Revision: D4824176
fbshipit-source-id: 5adca225d9984146852dad1e83de0d903848c1e5
Summary:
This simplifies the InodePtrImpl class by removing support for pointer-to-const
Inode objects. We don't currently use pointer-to-const Inode objects, and I
don't really anticipate that we will ever need this.
The primary motivating factor behind this diff is a recent change in clang that
breaks the existing code: clang PR31606 prevents children classes from
inheriting template constructor instantiations from their parent which take a
reference to the parent class type as the only argument.
While this could have been fixed by explicitly defining the necessary
constructors in the child class (or simply by overriding newPtrLocked() and
newPtrFromExisting() to return the subclass type), dropping support for
pointer-to-const allows us to simplify the code and resolve this issue.
Reviewed By: wez
Differential Revision: D4825526
fbshipit-source-id: 999de352788b13818b7f23788bb34686e193cd5d
Summary:
previously, the importer would read the entire manifest
and emit data to the store as it resolved complete directory entries.
The entire manifest data would be buffered and sent out to the store.
In the scenario where one subtree has been modified and a commit has
been made, only the parents of the subdirectory need to be hashed
and stored, but we would compute and try to store everything anyway.
While this diff can't avoid having to compute hashes for everything (we need
tree manifest data for that), by breaking the import into two passes we can
potentially avoid interrogating the LocalStore about every tree in the entire
manifest during an import; we only need to store the trees that are missing and
can simply cut out sub trees that are already present.
This saves us IO at the expense of buffering the manifest tree in memory
for the duration of the first pass; it is an acceptable trade off.
Reviewed By: simpkins
Differential Revision: D4832166
fbshipit-source-id: 0a40cb851c65393b407a8161db05c4b1795fb11a
Summary:
This avoids hard coding knowledge of the FB-specific `local` dir.
This is only suitable for use on already-mounted eden instances,
so the function accepts either the mounted path or the eden
configuration dir.
Reviewed By: bolinfest
Differential Revision: D4829166
fbshipit-source-id: df9daed6024cdec6222be90f5cb624dd9ca0da0b
Summary:
Folly is dropping support for GCC 4.8 now that the branch for last HHVM release that needs to support it has been cut.
This codemod's away all the trivially identifiable uses of `folly::make_unique` in fbcode outside of folly, mcrouter and hphp.
A future diff will kill it in those, and a diff after that will kill it entirely.
Reviewed By: djwatson
Differential Revision: D4498157
fbshipit-source-id: 5a2c05ffd29e5358298acb53dab189a699b022cc
Summary:
This diff fixes FileData::stat() so that we report reasonable timestamp values
on non-materialized files, rather than always leaving them as 0. We set the
timestamps to the time that we created the FileInode. This ensures that
timestamps are updated correctly when files are modified by a checkout
operation.
Note that for materialized files the code reports the timestamp of the overlay
file. This diff does not modify that behavior. However, this behavior is
incorrect, as the overlay file timestamps are not updated by a FUSE client
opening, modifying, then closing a file (since we keep the underyling overlay
file handle open, and don't close it).
In the future we'll need to implement our own tracking of atime, mtime, and
ctime values. We should probably store these in a header inside the overlay
file. For now, this diff is a stop-gap measure that ensures we at least update
non-materialized file timestamps correctly on checkouts.
Reviewed By: bolinfest
Differential Revision: D4765632
fbshipit-source-id: 478da6441e213cdfe830f1c5129212182ce4eeb0
Summary:
This diff adds a new "eden debug" command, with a few subcommands:
- "eden debug inode": Report information about currently loaded inodes
- "eden debug tree": Report data about source control Tree objects
- "eden debug blobmeta": Report metadata about source control Blob objects
- "eden debug blob": Report the contents of source control Blob objects
This diff also includes the thrift APIs in edenfs to support these commands.
Reviewed By: bolinfest
Differential Revision: D4760528
fbshipit-source-id: 7dc2bd6e0e952ba6fb61702c672fb9417d603ffa
Summary:
Previously EdenServer::getMount() returned a null pointer if the path did not
refer to a known mount point. However, most of the thrift APIs using this
method never checked if the return value was null, which could allow any thrift
client to crash edenfs.
This updates getMount() to throw if the path is not found, and never return
null.
Reviewed By: bolinfest
Differential Revision: D4760527
fbshipit-source-id: 6cd9a1e3bc0a0a220d7f5fa518d497d37f245eff
Summary:
This updates all of the references to gtest and gmock with googletest.
The change is mechanilcal, generated with the following one-liner:
```lang=bash
hg grep -lwE '(gtest|gmock)' 'glob:**/TARGETS' | grep -v '^third-party-buck' | xargs perl -pi -e '
$gt=qr!(["'"'"'])gtest\g1!;
(
s!$gt(\s*,\s*(.any.|None))(\s*,\s*)?\),?!\1googletest\1\2, \1gtest\1\),!g or
s!$gt((\s*,\s*(.any.|None)[^\)]+))\),?!\1googletest\1\2\),!g or
s!\(\s*$gt,?\s*\),?!\(\1googletest\1, None, \1gtest\1\),!g or
s!$gt,?!\(\1googletest\1, None, \1gtest\1\),!g
) unless /(name|type) *=/;
$gm=qr!(["'"'"'])gmock\g1!;
(
s!$gm(\s*,\s*(.any.|None))(\s*,\s*)?\),?!\1googletest\1\2, \1gmock\1\),!g or
s!$gm((\s*,\s*(.any.|None)[^\)]+))\),?!\1googletest\1\2\),!g or
s!\(\s*$gm,?\s*\),?!\(\1googletest\1, None, \1gmock\1\),!g or
s!$gm,?!\(\1googletest\1, None, \1gmock\1\),!g
) unless /(name|type) *=/;
'
```
Reviewed By: meyering
Differential Revision: D4643237
fbshipit-source-id: fda7f41760c7e44254231df87634631c343e6355
Summary:
Update the integration test code to allow TestCase classes to control the
--vmodule settings used when starting the edenfs daemon.
D3851805 initially set the vlog level for RequestData to 5 to help debug
issues. However, this log level doesn't really make sense for most of the
mercurial integration tests: most mercurial commands checks for the existence
of lots of files, causing the logs to be filled with useless ENOENT messages
when the RequestData log level is this high.
Reviewed By: wez
Differential Revision: D4814402
fbshipit-source-id: 1127c0a25b656ea1a710ca54a59a9407d66a5659
Summary:
Add an assert_status() method to the hg integration tests that runs "hg
status", parses the output, then compares it to expected results.
Reviewed By: wez
Differential Revision: D4814422
fbshipit-source-id: 24ebdc2e0239c4833953c31e5786cc320bcd9d62
Summary:
Update HgRepository.hg() to catch subprocess.CalledProcessError exceptions and
translate them into subclass that returns a more helpful message.
The CalledProcessError includes the command stderr as a member variable, but
unfortunately does not include this in the output from `__str__()`. The stderr
output is usually important for helping debug test failures.
Reviewed By: wez
Differential Revision: D4814401
fbshipit-source-id: 4f010bb3ce33833c55d18768997d2d1bb6a001b8
Summary:
The hg_import_helper script that eden uses to import data from mercurial keeps
a long-lived repository object open. This caches some data about the
repository, and if new commits are added after it was created, it can fail to
see them.
This updates hg_import_helper.py to catch errors that occur when trying to use
the repository objects. The code will invalidate the repository object and
then retry the operation once, in the hopes that it will now succeed after
invalidation.
Reviewed By: bolinfest
Differential Revision: D4752659
fbshipit-source-id: 1c75c84766d6bbda0710882a338eaa09e0cb0030
Summary:
The kernel can return ENOENT in response to invalidation notification if we
have never told the kernel about the inode in question. This resulted in
spurious errors during checkout when updating files that were loaded internally
by edenfs rather than via FUSE call. For instance, this was commonly triggered
by .gitignore files, which eden loads on its own to perform ignore processing.
Reviewed By: bolinfest
Differential Revision: D4752630
fbshipit-source-id: d4e092643a8d33cf33709f7e3664289f167ac093
Summary:
I found it rather awkward in HgExtensionTestBase that self.repo is not actually
the repository being tested. It was instead the repostiory used as the backing
store for the mercurial data, and self.repo_for_mount was the repository being
tested.
This diff renames the two repository classes, so that self.backing_repo is now
the backing store repository, and self.repo is the repository being tested.
In order to do this I changed HgExtensionTestBase to derive directly from
EdenTestCase. Previously it derived from EdenHgTest, and was letting
EdenHgTest set up self.repo. It seemed more understandable to avoid deriving
EdenHgTest now since self.repo is not the repository that needs to be set up
initially.
Reviewed By: bolinfest
Differential Revision: D4752631
fbshipit-source-id: d8b542b0ecead66b965af1a582085345e28b2908
Summary:
Previously the eden hg extension short-circuited the checkout operation if the
destination commit was the same as the one currently checked out. This was
incorrect if --clean was specified, since we do need to reset the working
directory state in this case.
This updates the extension code to always make the thrift checkout() call when
doing a force checkout.
This also avoids calling applyupdates() to resolve conflicts when force=True.
When doing a force checkout, eden reports files with conflicts that it
overwrote, but these do not need to be resolved by mercurial.
Additionally, this also updates a couple other APIs that have recently been
changed in upstream mercurial: merge.update() now takes an updatecheck
argument, and repo.join() should now be written as repo.vfs.join().
Reviewed By: bolinfest
Differential Revision: D4752510
fbshipit-source-id: e1ee92d086315e35a1378f674e668876a667c0ce
Summary:
Tweaks the stats stuff so that we can name the histograms and export them with p50, p90, p99 percentiles.
This is made a bit ugly because the fb303 stats code isn't yet open source, so
there's a moderately ugly preprocessor directive that we assume to be true for
our internal build. We'll need to force that to be 0 for the open source build
until we open the stats code and make it available.
Reviewed By: bolinfest
Differential Revision: D4801868
fbshipit-source-id: 643909e63bd4a74b2cfa580be131f65c5673bc94
Summary:
This explicitly uses a WriteBatch during importing. This results in a
5x perf gain for import operations.
I've also added in a read-before-write check to make sure that we're not
attempting to overwrite a key that already exists. Since this is a
content-addressed store, a given key should always have the same
value; it should be immutable. Overwriting an existing key is an
opportunity to change data that should never change and is also wasteful
of IO bandwidth.
Ideally we'd be able to exploit this to avoid descending into a directory;
I'd like to hook that up to the treemanifest stuff in mercurial in a
follow on diff.
Reviewed By: bolinfest
Differential Revision: D4783598
fbshipit-source-id: 24f95495d09a852859b346559fdf83d064de2b51
Summary:
this is the bare minimum to support creating unix domain sockets.
We only support using mknod to create a unix socket; other uses will yield an error.
I've added an rdev field as a sibling of the existing mode field that we track,
as that is the additional parameter that we need to track as part of the
special file node.
Special file nodes are tracked in the overlay as empty files.
Reviewed By: bolinfest
Differential Revision: D4774099
fbshipit-source-id: 0824b7e509063faa8bede7aff82a7c51930c4f83
Summary:
This is an initial pass at implementing "hg status" using the new EdenMount and
TreeInode diff() code.
More work still needs to be done to clean things up here, but this makes
"hg status" more correct in the face of modified files and directories that are
not modified, and vice-versa.
In particular, the following still needs to be cleaned up in future diffs:
- Update the "hg status" code to more correctly process user directives for
paths that did not appear in the diff output. This will probably be
simplified some by correctly updating the user directive list on checkout and
resetCommit() operations. It may also make things simpler if we store the
user directives in a hierarchical path map, rather than in a flat list.
- Update the addAll() code to also use the new diff logic, rather than the
getStatusForExistingDirectory() function.
- Clean up the locking some. I think it may be worth changing the code to use
a single lock to manage both the current commit ID for the mount and the
dirstate user directives. We probably should hold this lock for the duration
of a diff operation. The diff operation should also return the commit ID
that it is based against.
Reviewed By: wez
Differential Revision: D4752281
fbshipit-source-id: 6a6d7e2815321b09c75e95aa7460b0da41931dc0
Summary:
This is a utility that is intended to be consumed by our
fbagent monitoring infrastructure.
The typical pattern at FB is that fbagent is configured via monitoring config
to pull data from the hosts that belong to a thrift tier and then push that
into ODS.
Since our model is a bit different we don't fit that pattern; eden runs as a
process per user on a host, and is accessible only via unix domain socket.
That means that we need to use a CommandCollector to sample data from our eden
servers, and that's where this tool comes in.
The intent is to use configerator to set up the command collector to run the tool
using a command line that looks something like this:
```
eden-fb303-collector /data/users/*/.eden/socket --tierName %%TIER_NAME%% --counterRegexes '[".*mount.*.rate.60"]'
```
Note that we rely on the shell to expand the glob to find plausible eden sockets.
The `counterRegexes` parameter is a JSON encoded array listing the pcre regexes
that should be used to select the counters that we want to place into ODS.
I chose to pass these as JSON encoded parameters because it will allow the list
of counters to be configured easily in configerator (python) code alongside the
CommandCollector definition and pass that down to the collector. That makes it
a no-brainer to `json.dumps()` and pass the list down.
This collector emits three entities for each key:
* user@hostname - allows us to distinguish different users on the same host.
* hostname - aggregated by fbagent for all instances on the same host.
* TIER - aggregated by fbagent for all instances on the same tier.
I could add more levels of aggregation to this; it is common to track by region
and cluster, but I'm not sure how many of these we really want; any strong preferences?
The intent is that we'll deploy this alongside the edenfs binary that we
package with packman and then have a separate configerator diff to add the
CommandCollector to enable collection for the dev tiers.
D4797021 and D4797101 are the related diffs to hook this up in the right places
Reviewed By: bolinfest
Differential Revision: D4789717
fbshipit-source-id: 4a1d58c50847a8ca99043542cd2f0415da783ad7
Summary:
Leaving these as nullptr causes the kernel to handle advisory locking for us.
There's some plumbing in libfuse that handles negotiating this for us.
Reviewed By: simpkins
Differential Revision: D4774540
fbshipit-source-id: 5495963b980d95f06ededa72f4d483cc74fd8ec3
Summary:
It's not really magic because we don't have a virtual directory
inode base any more. Instead, we mkdir and populate it at mount time.
What is slightly magical about it is that we give it some special powers:
* We know the inode number of the eden dir and prevent unlink operations
on it or inside it.
* The .eden dir is present in the contents of the root inode and will
show up when that directory is `readdir`'d
* When resolving a child of a TreeInode by name, we know to return the
magic `.eden` inode number. This means that it is possible to `stat`
and consume the `.eden` directory from any directory inside the eden
mount, even though it won't show up in `readdir` for those child dirs.
The contents of the `.eden` dir are:
* `socket` - a symlink back to the unix domain socket that our thrift
server is listening on. This means that it is a simple
`readlink(".eden/socket")` operation to discover both whether a directory
is part of an eden mount and how to talk to the server.
* `root` - a symlink back to the root of this eden mount. This allows
using `readlink(".eden/root")` as a simple 1-step operation to find
the root of an eden mount, and avoids needing to walk up directory
by directory as is the common pattern for locating `.hg` or `.git`
dirs.
Reviewed By: simpkins
Differential Revision: D4637285
fbshipit-source-id: 0eabf98b29144acccef5c83bd367493399dc55bb
Summary:
Update the EdenMount::diff() code to completely skip .hg and .eden directories.
This was implemented through the GitIgnore code, and adding a new HIDDEN status
The .hg and .eden directories are similar to ignored directories, but we never
want to report any information for them even when listIgnored is true.
This also changes the GitIgnore code to so that GitIgnoreStack computes the
basename for each path once, and passes it down into the underlying match code,
so we don't have to re-compute it for each pattern that needs the basename.
Reviewed By: wez
Differential Revision: D4751917
fbshipit-source-id: ec22d62f31a3ce9ef998a8ccb4413f72f1d7a487
Summary:
This updates the diff() logic to process gitignore rules.
The bulk of the existing TreeInode::diff() was moved into a
TreeInode::computeDiff() function. TreeInode::diff() now loads the gitignore
rules for this directory before calling computeDiff(). The argument list for
computeDiff() was now getting slightly unwieldy, so I created a new DiffContext
class to hold some of the arguments that are the same for the duration of the
entire diff operation. This resulted in some rather mechanical changes through
the computeDiff() and DeferredDiffEntry code to update the argument lists.
Reviewed By: wez
Differential Revision: D4744690
fbshipit-source-id: c4981d9f49878b522e0b3faf35de837739066f3c
Summary:
Add suffixes() and rsuffixes() methods to the RelativePath and AbsolutePath
classes.
I am planning to use this functionality when performing gitignore matching.
(Each .gitignore file matches against relative path suffixes from the directory
containing the .gitignore file.)
Reviewed By: wez
Differential Revision: D4744612
fbshipit-source-id: 1fd3bdb4a66193fbd73aa514442c51b5024a30e3
Summary:
This tweaks the definition of the subscribe method in the
streamingeden thrift file and implements the server side of the
thrift service, and the client side to consume it in watchman.
The definition is now a bit simpler than it was previously; we're
now just sending a stream of the updated JournalPosition objects
to the client rather than computing and sending FileDelta objects.
This is cheaper for Eden to deal with because it is very cheap to take
the current journal position and pass that over the wire. This is
important because a burst of mutations will likely queue up a bunch
of these in quick succession.
In a future diff I'm considering adding a latency parameter for
the subscription so that we can constrain the number of updates
over a certain time period (likely in the 10's of milliseconds range).
For now I just want to prove that the concept works.
On the watchman side we just need to pull these off the wire as they are sent
by eden, then wait for the subscription stream to settle before internally
broadcasting to any connected subscribers.
Reviewed By: bolinfest
Differential Revision: D4647259
fbshipit-source-id: 03aa16e59a43195a2505a8d03bce1ccf88a8d42f
Summary:
Update the edenfs main function to check the exit code of the privhelper
process, and to exit with EX_SOFTWARE rather than EX_OK if the privhelper did
not exit with status 0.
This will ensure that our integration tests always catch crashes in the
privhelper process. Previously we could catch errors that happened before the
privhelper socket was closed, but if the privhelper server crashed during
cleanup we never noticed the error.
Reviewed By: bolinfest
Differential Revision: D4728006
fbshipit-source-id: 2c478612e5fabec90438b28b9c76a3343f561655
Summary:
Add a unit test that modifies a file and then does a forced checkout to revert
back to the original commit state.
This also includes some pretty-printers to make the tests print
CheckoutConflict objects nicely if any test assertions fail.
Reviewed By: bolinfest
Differential Revision: D4727779
fbshipit-source-id: b31dc90be28d17ed0f2a4076c96a08ba82588190
Summary:
This begins adding an EdenMount::diff() method, which walks through the inode
tree and reports differences from the current source control tree state.
My intent is to eventually update the Dirstate code to use this diff()
function, and to remove the existing getModifiedDirectories() logic. The
getModifiedDirectories() code is currently incorrect, as it reports
materialized directories, which is not the same as directories that are
modified from the current source control state.
The ignore processing logic is not currently implemented in EdenMount::diff(),
but it should be relatively straightforward to add in a subsequent diff.
Reviewed By: bolinfest
Differential Revision: D4726048
fbshipit-source-id: ad0bb3b5d72bb3830f60fc2b2e56a81217c35353
Summary:
The privhelper code was erasing elements from a std::unordered_map before it
was done using the iterator pointing to that element. This causes memory
corruption issues.
Between this and some of my other recent unmount fixes (D4548030, D4547938)
this makes the bind-mount-related integration tests work.
Reviewed By: bolinfest
Differential Revision: D4727850
fbshipit-source-id: 6d1fda3f89cb91c89d0020921b1805fc10e65785
Summary:
Update the TreeInode and CheckoutAction to correctly be able to remove
directories during checkout operations. This handles both removing directories
that are newly empty as a result of the checkout, and directories that are
replaced with a file or symlink.
This also refactors the rmdir/unlink code a bit so we can share some of its
logic for doing directory removal during the checkout operation.
Reviewed By: bolinfest
Differential Revision: D4708607
fbshipit-source-id: 3b9dd9cc3536845dad0d9278e3c5d3ac1ed04570
Summary:
Synchronize access to the CheckoutContext conflicts_ list when recording
conflicts or errors during a checkout operation.
In theory it is possible for the checkout operation to be proceeding
simultaneously in multiple threads. At the moment this does not happen because
the BackingStore implementations currently block, and always return
immediately-ready Future objects in the same thread. However, once they are
updated to return Future objects that actually complete asynchronously in other
threads the checkout work will continue in these other threads.
Reviewed By: bolinfest
Differential Revision: D4708605
fbshipit-source-id: 885b0114d97dadcacf64652f795a7d3846412f11
Summary:
Add a test that exercises a checkout operation that creates a brand new
subdirectory.
Reviewed By: bolinfest
Differential Revision: D4693994
fbshipit-source-id: 70f43f67c2cdb10bd854f51d5ba2e61c45976975
Summary:
Unfortunately it looks like gcc won't support the [[nodiscard]] attribute until
the gcc-7.x release. (The functionality is available in gcc-4.8 and later, but
using the name [[gnu::warn_unused_result]] instead.)
This changes the code to use the FOLLY_WARN_UNUSED_RESULT macro from
folly/Portability.h instead. (This expands to older non-standard __attribute__
syntax for gcc and clang, and _Check_return_ on Windows.)
Reviewed By: bolinfest
Differential Revision: D4704184
fbshipit-source-id: d5c13630ea611a2f43080a501add42ce9fda6789
Summary:
Add new ASSIGN_PARENT_INODE and ASSIGN_INODE types to LoadBehavior, to exercise
additional code paths during checkout operations. This also cleans up the
overall handling of the LoadBehavior enum.
Reviewed By: bolinfest
Differential Revision: D4694548
fbshipit-source-id: e8359cfdadb1fabb1a0d07bc57a1a1610ed9ba57
Summary:
Refactor FileInode and FileData so that they no longer store the
TreeInode::Entry that refers to this file. The TreeInode::Entry is owned by
the parent TreeInode and is not safe to access without holding the TreeInode
contents_ lock, which the FileInode code never did. Additionally, once a
FileInode is unlocked the TreeInode::Entry is destroyed. Previously the
FileInode code would retain its pointer to this now invalid data, and could
still dereference it.
In the future I think it might be worth refactoring the FileInode/FileData
split a bit more, but this at least addresses the issues around invalid
accesses to entry_.
Reviewed By: bolinfest
Differential Revision: D4688058
fbshipit-source-id: 17d5d0c4e756afc6e3c4bb279cb1bb5231f3134f
Summary:
Remove the TreeInode::entry_ member variable. This was a pointer to the
TreeInode::Entry object representing this inode in the parent TreeInode.
However, it isn't safe for a child Inode to access the TreeInode::Entry owned
by its parent. This Entry object is supposed to be protected by the parent
TreeInode's contents_ lock, and the child inode never held this when accessing
the Entry.
This updates the TreeInode code to never use entry_ any more. I will work on
removing FileInode::entry_ in a subsequent diff.
This did require changing the overlay materialization code some. Previously
the checkout code directly modified entry_ to inform the parent TreeInode if
the child was materialized or not. Now we have to more explicitly tell the
parent TreeInode about changes to each child's materialization state. With
this change we now keep the overlay data on disk more consistent during the
checkout process. We should now always have overlay data for children inodes
on disk whenever the parent thinks they are materialized. We may end up
writing out TreeInode overlay data frequently during the checkout process,
however. We might be able to improve this in the future. For now it seemed
simpler and safer to just update the data frequently to make sure it stays
consistent.
Reviewed By: bolinfest
Differential Revision: D4688055
fbshipit-source-id: d08a14b9304ac49826f0897a0b53281177d9d9f4
Summary:
Update the code so that we no longer immediately remove overlay data about a
file or directory when the inode is unlinked. The file can continue being used
after it was unlinked if there are open file handles or other internal
references to it.
This updates the code to instead wait to remove the overlay data until the
InodeBase object is unloaded.
This does not fully fix all the outstanding issues with using inodes after they
have been unlinked, because the FileInode and TreeInode still use the entry_
pointer that now points to a bogus TreeInode::Entry.
Reviewed By: bolinfest
Differential Revision: D4688056
fbshipit-source-id: a9e1c1a9562590b8b81237057bfc4ef77795807f
Summary:
This changes a couple aspects of the materialization process:
- Write out child files in the overlay before their parent. If we throw an
exception or crash partway through this process, it is better to have the
child not marked materialized in its parent, rather than to have the parent
indicate that the child is materialized but to not actually have any overlay
data for the child. In the former case we will still load the correct child
data the next time we need to, but in the latter case we would fail to load
the child correctly.
- Hold the rename lock while materializing our parent directories, to ensure
that we materialize the correct parent, and it cannot change while we are
trying to perform the materialization.
- Make the parent responsible for modifying the child's TreeInode::Entry. This
data is owned by the parent and is protected by the parent's contents_ lock.
The child really shouldn't ever be directly accessing this data without
holding the parent's contents_ lock.
Reviewed By: bolinfest
Differential Revision: D4688057
fbshipit-source-id: 2662f79cb7d7febb086f4e0888a3d96a580c4bfa
Summary:
This fixes the checkout code to always recurse into directories with conflicts,
and accurately list all individual files with conflicts.
Previously the checkout code always avoided recursing into unloaded directory
inodes that were affected by the checkout. Since the inode was not loaded we
can easily replace it without recursing into it. Unfortunately this resulted
in an inaccurate conflict list--we would report the directory itself as a
conflict. This is insufficient for our mercurial extension, which needs to
know the specific list of files with conflicts, so it can ask mercurial to
perform conflict resolution on them.
Reviewed By: bolinfest
Differential Revision: D4672373
fbshipit-source-id: ff400a4a6e25b8b0754c66cb067a69d553a5c98b
Summary:
Loading the child inode is necessary to avoid race conditions with other
threads that may be currently trying to load the child inode in question.
This updates the code so that rmdir() and unlink() now share most of their
implementation.
Reviewed By: bolinfest
Differential Revision: D4664110
fbshipit-source-id: 871ca7b3c22819b24bc2639d9550bb45f6541f50
Summary:
Update the DEFS and .buckconfig files so that the C++ code can be built
successfully with buck in our github repository.
Reviewed By: wez
Differential Revision: D4633749
fbshipit-source-id: a07a6e664471237fcc2d13a8be8b92efa3a101bf
Summary:
Replace the one very basic checkout unit test with several tests that
more thoroughly test file modification changes. More tests for
directory changes and file type changes still need to be added, but this is a
first step for now. I will continue adding more tests in upcoming diffs.
Reviewed By: wez
Differential Revision: D4641595
fbshipit-source-id: 443a68e4e3c673fe687446f3469f035b4f48f1b3
Summary:
Add a method to reset the current commit without changing the working
directory state, similar to "hg reset" and "git reset --soft".
This also adds a new EXPECT_FILE_INODE() check for use in the unit tests,
and FileInode::getPermissions() and FileData::readFull() methods to support
this check.
Reviewed By: wez
Differential Revision: D4641476
fbshipit-source-id: 1e516774fe8e292a8d82cc2c354619374a3abe37
Summary:
The FakeTreeBuilder class in D4609587 provides a lot of the same functionality
as the TestMountBuilder class, but is not restricted to being used only at
mount initialization time. (It also provides more powerful functionality for
controlling the ordering of events when loading data from the ObjectStore.)
This switches all of the existing tests to use FakeTreeBuilder rather than
TestMountBuilder, and removes the TestMountBuilder class.
One difference is that FakeTreeBuilder adds data to the FakeBackingStore, while
TestMountBuilder previously put data directly into the LocalStore. This
shouldn't really make much difference for the behavior of the code. Putting
data in the FakeBackingStore gives us more control over when data is available
during load operations, and exercises slightly more of the normal ObjectStore
code path.
Reviewed By: wez
Differential Revision: D4641288
fbshipit-source-id: ca2b45bf69bd373848c12a7b739b286aabe6aa9b
Summary:
Add a FakeTreeBuilder class to make it easier to create Tree structures in the
FakeBackingStore using path names. Previously the test code had to manually
chain together Trees and Blobs. FakeTreeBuilder lets callers call setFile()
with a path name to define the file layout. The files that were defined then
get converted into Tree and Blob data when finalize() is called.
This is similar to the existing TestMountBuilder::addFile() API. However,
TestMountBuilder only populates the LocalStore. FakeTreeBuilder populates
the FakeBackingStore instead. The FakeBackingStore class allows tests to have
much more control about when objects appear ready from the backing store,
allowing control over the order in which Futures are fulfilled during the test.
Reviewed By: bolinfest
Differential Revision: D4609587
fbshipit-source-id: 9642d17daf0dc10f08901e51552bbb3c3e150b53
Summary:
Make sure the eden coded compiles cleanly with -Wshadow-compatible-local
Pretty much all of the warnings were issues with lambdas shadowing names from
their parent context (even though they didn't ask to capture those names from
the parent).
Reviewed By: wez
Differential Revision: D4644849
fbshipit-source-id: 66629cd98b5af4760f3fbb256e44c0bc47e52316
Summary:
Previously, we would only allow you to consume a symlink that was
checked into a revision, but not allow you to create a new one.
This diff adds support for making new symlinks.
It turns out that the code that we had for readlink had an unused code path for
reading an actual symlink out of the overlay. My first pass at this diff tried
to make the materialization code generate such a symlink, but it was breakin
some other assumptions in the rest of the code.
This version of the code just stores the symlink target in the file contents.
This means that we can simplify readlink to just calling `readAll` on the
underlying file data, and that will load the contents out of the storage layer
if the file wasn't materialized. This helps simplify things a bit more.
Reviewed By: bolinfest, simpkins
Differential Revision: D4604016
fbshipit-source-id: 3138d0f9880639d2bbeaf4bb03bef3f021c3ecb3
Summary:
In order to send realtime subscription information to watchman
we need to use the thrift streaming data functionality. Since this
isn't supported in java we originally broke this out into its own
thrift file.
It's now time to hook this up; this diff adds some plumbing so that
our thrift server knows to implement this interface.
This was made a bit more complicated because the inheritance in
the thrift definitions causes some ambiguities in the header maps
that buck uses to manage the includes.
I also had to rename the file from `eden.stream.thrift` to `streamingeden.thrift`
to avoid a problem with the period in the middle of the identifier.
For the server side of things, I've removed the `:thrift-cpp2` target
and replaced it with `thrift-streaming` instead.
This diff doesn't implement the endpoint, it is just setting things
up for the follow-on diff that does.
Reviewed By: bolinfest
Differential Revision: D4640639
fbshipit-source-id: b3625b0ff33a42945bf523944beed050f549a18c
Summary:
I originally added this to facilitate `hg status` but we didn't end up
using it. Rather than keeping it around and updating the tests in the `.eden`
dir diffs that follow, let's just remove it.
Reviewed By: simpkins
Differential Revision: D4610768
fbshipit-source-id: 158c0207f88980e86aeeddf75e6fd49763d2a402
Summary:
Remove AutoHeaders.RECURSIVE_GLOB from all of our TARGETS files. This is a
Facebook-internal directive that was added as part of a codemod. The vanilla
open source buck does not know about this directive, and can't parse TARGETS
files that contain it.
All of the affected rules are cpp_unittest() and cpp_binary() rules that don't
actually have any header files, so these parameters didn't actually have any
effect.
Reviewed By: andrewjcg
Differential Revision: D4633748
fbshipit-source-id: 6227fe9b7f2ea838a0c72408ca195685bcae9f5a
Summary:
Add components() and rcomponents() methods to ComposedPathBase.
These functions return a pair of iterators allowing you to iterate through the
components in the path.
This is similar to the existing paths() and rpaths() method which iterate
through path prefixes.
Reviewed By: wez
Differential Revision: D4609907
fbshipit-source-id: 698e5d58db897bbc760f2a2de90ef430fcb03a52
Summary:
In TreeInode::computeCheckoutActions() the code incremented the wrong index
when we reached the end of the new tree before the old tree.
panda-facepalm
Reviewed By: bolinfest
Differential Revision: D4609906
fbshipit-source-id: 61ca6baabe4eebb033c48b742562558466a45104
Summary:
Invalidate directory entries as they are modified during the checkout
operation. We never need to invalidate inode contents, since we always update
files by replacing the file with a new inode.
Reviewed By: wez
Differential Revision: D4577942
fbshipit-source-id: afde3040960b076d2cd384301a89cd7bdfce5a6a
Summary:
When the "eden daemon" CLI command has to run edenfs via sudo, also explicitly
pass through the KRB5CCNAME environment variable. This will allow edenfs and
it's hg import helper to find the correct Kerberos credentials if it needs to
connect to a remote server.
Reviewed By: wez, bolinfest
Differential Revision: D4584132
fbshipit-source-id: 6c72e4eca76d81b3d4b53a9e72a0e8388da30a7d
Summary:
This updates CheckoutAction to make the old TreeEntry object optional.
This is necessary to support situations where the entry did not exist in the
old source control tree, but does exist in the new tree and also exists locally
on the file system. This situation is always a conflict, so it only applies
when a force checkout is being done.
This diff also refactors the TreeInode::computeCheckoutActions() logic to
combine the processNewEntry/processRemovedEntry/processEntry lambdas into a
single function, as suggested in the code review for D4538516.
Reviewed By: wez
Differential Revision: D4577924
fbshipit-source-id: 9435205b541a12c75a8b9bd2b2b599a339a2f6c8
Summary:
Add a thrift call to get the ID of the snapshot currently checked out in a
given eden mount.
Reviewed By: bolinfest
Differential Revision: D4570729
fbshipit-source-id: 208ceb309aacfb07e30ddb8fdfc0950a3d7d748b
Summary:
Fix the FileHandle code that was calling getPathBuggy() when writing journal
entries. This fixes a crash when writing to an unlinked file.
Also remove an unused call to getPathBuggy() in TreeInodeDirHandle.
These were the last places calling getPathBuggy(), so I have now deleted this
method entirely.
Reviewed By: bolinfest
Differential Revision: D4570606
fbshipit-source-id: a1c4aab10df96e2aeee78fca6d5808db28ad0cbf
Summary:
When the privhelper process detects that the connection to the main eden
process has closed, have it forcibly unmount the eden mount points.
(This does the equivalent of "umount -l")
This is needed to properly clean up mount points which contain bind mounts
inside them. If eden has already exited we will not be able to unmount the
bind mounts, and therefore we won't be able to unmount the parent eden mount
directory either without using MNT_DETACH.
In the long run we will probably want to update the eden process to shut down
gracefully on SIGINT or SIGTERM. This would let us unmount everything cleanly
if we did the unmount operations before we stop responding to FUSE requests.
However, the privhelper still needs to be able to clean things up properly if
eden crashes, so this lazy unmount behavior will still be desirable in the long
run.
Reviewed By: wez
Differential Revision: D4548030
fbshipit-source-id: b60189181bccc6f126998c92fd9a2e47593c96b3
Summary:
In the privhelper code we want to wait for the bind unmount to really complete
before proceeding. Previously this was done by repeatedly calling statfs()
until it fails. However, on my system the statfs() call continues to succeed
even after the unmount operation, causing bindUnmount() to never think that the
unmount has finished. (I'm currently running Linux 4.6.7)
This updates the code to check the file system ID before and after the unmount,
and also stop once we see the filesystem ID change. I also replaced the
Linux-specific statfs() call with the POSIX-standard statvfs() call.
Reviewed By: wez
Differential Revision: D4547938
fbshipit-source-id: 51f53c0c901331de80fb908ae30033a0b56091b4
Summary:
This is the initial code for implementing checkout.
This isn't quite 100% implemented yet, but I think it's worth checking in this
code as-is, and getting the remaining functionality done in separate diffs.
In particular, a few operations aren't implemented:
- Removing a directory that was deleted in the new revision
- Replacing a directory that was replaced with a file or symlink in the new
revision
- When doing a forced update, replacing a file or directory that did not exist
in the old revision, but that was created locally in the working directory,
and also exists in the new revision.
Reviewed By: wez
Differential Revision: D4538516
fbshipit-source-id: 5bb4889b02f23ab2048fcae2c8b7614340181aa6
Summary:
If the eden process times out waiting on a response from the privhelper, the
next time it tries to send a request the old response may be available now.
In this case it will read the old response when expecting to read the response
for its new request.
This diff updates the code to discard the old response in this case and retry
reading another request. This allows us to get back in sync with the
privhelper after a timed-out request. Previously the code would simply give up
after finding a mismatching transaction ID in the response, causing it to
always be one response off, and preventing any future requests from succeeding.
This is still kind of hacky, and could definitely be improved in the future,
but for now it is better than always failing all future privhelper requests.
Reviewed By: wez
Differential Revision: D4547769
fbshipit-source-id: 396b57eccb28d938998e29594d336f35ba4fb1fb
