Summary: It doesn't need to exist anymore
Reviewed By: yfeldblum
Differential Revision: D5318746
fbshipit-source-id: c70b184f4b3fc12ede4632d6b3d43de16ed758c7
Summary:
this code was present to help in some benchmarking
in the very early prototype code. I noticed that this was
still lingering when reviewing a diff the other day.
Remove it!
Reviewed By: simpkins
Differential Revision: D5334263
fbshipit-source-id: 9ecccf1f9922b4c8f46b2529da665e2fdf11ab7a
Summary:
Format all of the TARGETS files under eden/fs with the autodeps tool.
A few rocksdb include statements require comments so that autodeps can
correctly tell which dependency this include comes from. The rocksdb library's
source file structure unfortunately does not match the layout of how its header
files get installed, so autodeps cannot figure this out automatically.
Reviewed By: wez
Differential Revision: D5316000
fbshipit-source-id: f8163adca79ee4a673440232d6467fb83e56aa10
Summary:
Update EdenServer to catch SIGTERM and SIGINT, and to stop gracefully on these
signals.
Reviewed By: wez
Differential Revision: D5315323
fbshipit-source-id: 16611190c8d522a78cf6b624d97d4f7ecc478f96
Summary: Add unit tests for HgImporter to better test this import logic.
Reviewed By: bolinfest
Differential Revision: D5309171
fbshipit-source-id: b5619d8271fef1cc31a0f642ec2fcbf9eccced54
Summary: This includes adjusting them to conform to Folly's style for single file sources and headers, making targets as small as possible, not providing aggregate targets, and using auto-deps.
Reviewed By: yfeldblum
Differential Revision: D5312078
fbshipit-source-id: 1c7f5aa04da3bad236ffa23000c7bda47b82e3ef
Summary:
Update the findImportHelperPath() function to search all parent directories
containing the binary for the hg import helper script, rather than hard-coding
exactly how many directories deep we expect the edenfs binary to be placed in
the build output directory.
This makes it possible to use HgImporter in unit tests and other binaries that
aren't built in the exact same directory as the main edenfs binary.
Reviewed By: bolinfest
Differential Revision: D5309170
fbshipit-source-id: 051b26ea3017d8a9a2b9e2c59e164c28b640d1f5
Summary:
Update HgBackingStore and GitBackingStore to accept the repository path as an
AbsolutePathPiece rather than as a plain StringPiece.
Reviewed By: bolinfest
Differential Revision: D5309172
fbshipit-source-id: aca4818c3add11d07ece796298f6175ca4fb1448
Summary:
1. Moved read, write, mkdir, rm methods in hg/lib/hg_extension_test_base.py to lib/test_case.py.
2. Added integration test case to test unload free inodes.
Reviewed By: simpkins
Differential Revision: D5277870
fbshipit-source-id: b93b6049a10357cf8c92366e6dca3968f7f30c30
Summary:
Update eden to log via the new folly logging APIs rather than with glog.
This adds a new --logging flag that takes a logging configuration string.
By default we set the log level to INFO for all eden logs, and WARNING for
everything else. (I suspect we may eventually want to run with some
high-priority debug logs enabled for some or all of eden, but this seems like a
reasonable default to start with.)
Reviewed By: wez
Differential Revision: D5290783
fbshipit-source-id: 14183489c48c96613e2aca0f513bfa82fd9798c7
Summary:
Now that the non-future versions of these APIs have been removed, rename
getBlobFuture() to getBlob(), and getTreeFuture() to getTree()
Reviewed By: wez
Differential Revision: D5295690
fbshipit-source-id: 30dcb88854b23160692b9cd83a632f863e07b491
Summary:
Remove the blocking getBlob() API.
There were a few call sites in FileData still using this blocking API. For now
I simply updated them to use getBlobFuture() and make a blocking get() call on
the returned future. These call sites already had TODO comments documenting
the blocking behavior.
I plan to rename getBlobFuture() to getBlob() in a subsequent diff.
Reviewed By: wez
Differential Revision: D5295726
fbshipit-source-id: 748fd7a140b9b59da339a330071f732bba38cb35
Summary:
Remove the blocking getTree() API. All call sites are using getTreeFuture()
instead now.
I plan to rename getTreeFuture() to getTree() in a subsequent diff.
Reviewed By: wez
Differential Revision: D5295725
fbshipit-source-id: 6b40b4c808da94a9c68decae3ce38c7d13fbe9f5
Summary:
Remove the ObjectStores.h and .cpp files. These files contained helper
functions for looking up Tree and TreeEntry objects based on a deep relative
path.
These functions are no longer used anywhere anymore (the code that performs
deep lookups always does inode lookups instead). Additionally, this file is
the only place still using some of the blocking, non-Future-based APIs for
object lookups. Deleting these files will let us remove these deprecated
blocking APIs.
Reviewed By: wez
Differential Revision: D5295691
fbshipit-source-id: 89229827305490eba4d3a581954a90c5cf63238d
Summary:
1.Added a new thrift method to unload free inodes of a directory and its sub directories.
2.Added a new debug sub command 'eden debug unload <path>' to cli tools to unload free inodes.
Reviewed By: simpkins
Differential Revision: D5261038
fbshipit-source-id: 85b4c5ae18c0ae24c666a44ac9892765e753397f
Summary:
This is a major change to Eden's Hg extension.
Our initial attempt to implement `edendirstate` was to create a "clean room"
implementation that did not share code with `mercurial/dirstate.py`. This was
helpful in uncovering the subset of the dirstate API that matters for Eden. It
also provided a better safeguard against upstream changes to `dirstate.py` in
Mercurial itself.
In this implementation, the state transition management was mostly done
on the server in `Dirstate.cpp`. We also made a modest attempt to make
`Dirstate.cpp` "SCM-agnostic" such that the same APIs could be used for
Git at some point.
However, as we have tried to support more of the sophisticated functionality
in Mercurial, particularly `hg histedit`, achieving parity between the clean room
implementation and Mercurial's internals has become more challenging.
Ultimately, the clean room implementation is likely the right way to go for Eden,
but for now, we need to prioritize having feature parity with vanilla Hg when
using Eden. Once we have a more complete set of integration tests in place,
we can reimplement Eden's dirstate more aggressively to optimize things.
Fortunately, the [[ https://bitbucket.org/facebook/hg-experimental/src/default/sqldirstate/ | sqldirstate ]]
extension has already demonstrated that it is possible to provide a faithful
dirstate implementation that subclasses the original `dirstate` while using a different
storage mechanism. As such, I used `sqldirstate` as a model when implementing
the new `eden_dirstate` (distinguishing it from our v1 implementation, `edendirstate`).
In particular, `sqldirstate` uses SQL tables as storage for the following private fields
of `dirstate`: `_map`, `_dirs`, `_copymap`, `_filefoldmap`, `_dirfoldmap`. Because
`_filefoldmap` and `_dirfoldmap` exist to deal with case-insensitivity issues, we
do not support them in `eden_dirstate` and add code to ensure the codepaths that
would access them in `dirstate` never get exercised. Similarly, we also implemented
`eden_dirstate` so that it never accesses `_dirs`. (`_dirs` is a multiset of all directories in the
dirstate, which is an O(repo) data structure, so we do not want to maintain it in Eden.
It appears to be primarily used for checking whether a path to a file already exists in
the dirstate as a directory. We can protect against that in more efficient ways.)
That leaves only `_map` and `_copymap` to worry about. `_copymap` contains the set
of files that have been marked "copied" in the current dirstate, so it is fairly small and
can be stored on disk or in memory with little concern. `_map` is a bit trickier because
it is expected to have an entry for every file in the dirstate. In `sqldirstate`, it is stored
across two tables: `files` and `nonnormalfiles`. For Eden, we already represent the data
analogous to the `files` table in RocksDB/the overlay, so we do not need to create a new
equivalent to the `files` table. We do, however, need an equivalent to the `nonnormalfiles`
table, which we store in as Thrift-serialized data in an ordinary file along with the `_copymap`
data.
In our Hg extension, our implementation of `_map` is `eden_dirstate_map`, which is defined
in a Python file of the same name. Our implementation of `_copymap` is `dummy_copymap`,
which is defined in `eden_dirstate.py`. Both of these collections are simple pass-through data
structures that translate their method calls to Thrift server calls. I expect we will want to
optimize this in the future via some client-side caching, as well as creating batch APIs for talking
to the server via Thrift.
One advantage of this new implementation is that it enables us to delete
`eden/hg/eden/overrides.py`, which overrode the entry points for `hg add` and `hg remove`.
Between the recent implementation of `dirstate.walk()` for Eden and this switch
to the real dirstate, we can now use the default implementation of `hg add` and `hg remove`
(although we have to play some tricks, like in the implementation of `eden_dirstate.status()`
in order to make `hg remove` work).
In the course of doing this revision, I discovered that I had to make a minor fix to
`EdenMatchInfo.make_glob_list()` because `hg add foo` was being treated as
`hg add foo/**/*` even when `foo` was just a file (as opposed to a directory), in which
case the glob was not matching `foo`!
I also had to do some work in `eden_dirstate.status()` in which the `match` argument
was previously largely ignored. It turns out that `dirstate.py` uses `status()` for a number
of things with the `match` specified as a filter, so the output of `status()` must be filtered
by `match` accordingly. Ultimately, this seems like work that would be better done on the
server, but for simplicity, we're just going to do it in Python, for now.
For the reasons explained above, this revision deletes a lot of code `Dirstate.cpp`.
As such, `DirstateTest.cpp` does not seem worth refactoring, though the scenarios it was
testing should probably be converted to integration tests. At a high level, the role of
`DirstatePersistence` has not changed, but the exact data it writes is much different.
Its corresponding unit test is also disabled, for now.
Note that this revision does not change the name of the file where "dirstate data" is written
(this is defined as `kDirstateFile` in `ClientConfig.cpp`), so we should blow away any existing
instances of this file once this change lands. (It is still early enough in the project that it does
not seem worth the overhead of a proper migration.)
The true test of the success of this new approach is the ease with which we can write more
integration tests for things like `hg histedit` and `hg graft`. Ideally, these should require very
few changes to `eden_dirstate.py`.
Reviewed By: simpkins
Differential Revision: D5071778
fbshipit-source-id: e8fec4d393035d80f36516ac050cad025dc3ba31
Summary:
This is generated by applying clang-tidy `-checks=modernize-use-override` to all the c++ code in project eden.
It enforces the use of the keywords `virtual`, `verride` and `final` in a way compliant to the style guide.
Reviewed By: igorsugak
Differential Revision: D5108807
fbshipit-source-id: 596f2d73f1137de350114416edb1c37da5423ed5
Summary: The `nodiscard` attribute was added to standard C++ in C++17. Prefer the standard formulation when it is available; otherwise, use `__attribute__((__warn_unused_result__))` on compilers that support the GNU extensions, and `_Check_return_` on MSVC. The old `FOLLY_WARN_UNUSED_RESULT` is now expands to `FOLLY_NODISCARD`.
Reviewed By: yfeldblum
Differential Revision: D5105137
fbshipit-source-id: 9aa22e81cd9f0b89f9343433aeae3ba365227ccb
Summary:
We're using `window` to constrain resources and maintain an upper
bound on the number of futures that we spawn. Ironically, for a largish number
of already completed futures, this results in a recursive chain of `spawn`
calls in the `window` implementation and results in a stack overflow (the kind
that crashes a program, not the kind that answers programming questions).
This diff removes the use of `window` and replaces it with `collect`.
An earlier iteration of this diff just made all of these calls serially,
blocking the main thread. This at least does things in parallel; both
can torture the system for pathological walks, but at least the parallel
one should complete faster.
Reviewed By: simpkins
Differential Revision: D4932774
fbshipit-source-id: 40d39a85029f38ff69a530070efb879a81950451
Summary:
It is an error to not provide mode argument when open is
called with O_CREAT.
Reviewed By: oebeling
Differential Revision: D5020517
fbshipit-source-id: 87665ceb16ebdff797f8bf3d33bea082191e3121
Summary:
On OSX mercurial libraries are not installed in the standard pythonpath. This
option lets me pass a pythonpath to the hg importer script only, without
polluting the pythonpath for other subprocess invocations (such as python 3
based hooks, which are not happy with python2 pythonpaths).
Reviewed By: bolinfest
Differential Revision: D4992222
fbshipit-source-id: aa8e734970344215cc8783e164fd119c3ee6aed6
Summary:
These files are defining flags, so they should import gflags.
This import being missing causes open source build to fail.
Reviewed By: bolinfest
Differential Revision: D4992246
fbshipit-source-id: 61eb35b43ae6f9ff88bd34a9ebc8db6d80ac9d08
Summary:
Update Dirstate::remove() to directly call TreeInode::unlink() instead of going
through the EdenDispatcher. Internal code should be able to use TreeInode
directly without going through the FUSE Dispatcher. This prevents us from
having to look up the TreeInode object by inode number again.
This also fixes TreeInode::unlink() and TreeInode::rmdir() to perform FUSE
cache invalidation correctly. EdenDispatcher used to do this for unlink()
calls, but it was missing for rmdir() calls.
Reviewed By: wez
Differential Revision: D4968832
fbshipit-source-id: 51fda5de196392c640436ca6ad7d8333e6799db9
Summary:
This replaces the shouldFileBeDeletedByHgRemove() function with a much simpler
and safer check.
It turns out this code was more complicated than it needed to be: it looked up
the inode in question and confirmed that it still existed, even though it's
calling function had already previously looked up the inode object.
I believe this deletes the last place in the code that still assumes that an
InodeBase object must be loaded if the inode has previously been materialized.
This also removes one of the last remaining places that directly holds the
TreeInode contents_ lock outside of TreeInode.cpp.
Reviewed By: bolinfest
Differential Revision: D4968834
fbshipit-source-id: a37bc0d7889eb81cca5b045cbc82732a53ef53d4
Summary:
This updates the Dirstate::addAll() code to use EdenMount::diff() internally,
instead of getStatusForExistingDirectory().
This lets us delete getStatusForExistingDirectory() and several other helper
functions that it used. The logic in getStatusForExistingDirectory() was based
on the incorrect assumption that files can only be modified from their expected
source control state if they are materialized. This could result in incorrect
results in a variety of cases (particularly after renames, or "hg reset"
operations). The EdenMount::diff() code does not have this problem.
That said, this new addAll() code does still have some performance issues--it
currently does a full tree diff for each input path. We should ideally fix
this in the future to only diff the necessary subtree for each path. However,
in the short term this trade-off seems worth being able to delete all of this
older, buggy code. diff() should be cheap enough in most cases that this won't
be a major problem unless a large number of paths are given as input.
Reviewed By: bolinfest
Differential Revision: D4968835
fbshipit-source-id: 1834aa98a26dcaa0e1c06c7ac25c57944fa1b5f7
Summary:
Make sure we use pointers to const GitIgnoreStack objects when possible.
Also change the code to store the root GitIgnoreStack object for a diff()
operation in the DiffContext object, instead of having EdenMount::diff()
allocate it separately on the heap. This will make it easier to consolidate a
bit more logic in the DiffContext class in the future. In particular, I
suspect we will want a version of diff() that only works on one portion of the
tree. Putting more of the functionality in DiffContext will make it slightly
easier to share code between the full-mount diff and the subtree diff
functions. This also happens to save a memory allocation for now.
Reviewed By: wez
Differential Revision: D4968833
fbshipit-source-id: 1dc33b3d44cdf00e93b22d810c3a736d27c13638
Summary:
Update the eden thrift APIs to support multiple thrift parents, and update the
hg extension to use the new thrift APIs.
Reviewed By: bolinfest
Differential Revision: D4949139
fbshipit-source-id: 98a7bd17ccad7be6a429df34ecfaf3fe7ae0b39a
Summary:
This updates the ClientConfig and EdenMount code to support storing two parent
commits.
This changes the on-disk SNAPSHOT file contents add an 8-byte header that
includes a file identifier and a file format version number, followed by up to
two commit hashes. The code currently can read either the old or new format
from the SNAPSHOT file. We should be able to drop the code for reading the old
format fairly soon if we want, though.
This diff only updates the ClientConfig and EdenMount code, and does not yet
update the thrift APIs or the eden mercurial extension yet. I will update the
rest of the code in a subsequent diff.
Reviewed By: bolinfest, wez
Differential Revision: D4943917
fbshipit-source-id: cf456e67b845aa0cf8b45c822985cb932df107b4
Summary: Add some unit tests that check the behavior of setattr() and getattr().
Reviewed By: bolinfest, wez
Differential Revision: D4941215
fbshipit-source-id: 6691f51aea742d0159a16112ca291546a8adc928
Summary:
The error messages for various failure cases in EXPECT_THROW_RE() and
EXPECT_THROW_ERRNO() were missing a space after the statement being executed.
Reviewed By: bolinfest
Differential Revision: D4941175
fbshipit-source-id: e10b4bff7db027acc5cc7ad5c12af248d55c7cd8
Summary:
This fixes "hg commit" so that it correctly updates the in-memory snapshot.
This has been broken ever since I added the in-memory snapshot when
implementing checkout(). The existing scmMarkCommitted() method updated only
the Dirstate object and the on-disk SNAPSHOT file.
This diff fixes checkout() and resetCommit() to clear the Dirstate user
directives correctly, and then replaces calls to scmMarkCommitted() with
resetCommit().
Reviewed By: bolinfest
Differential Revision: D4935943
fbshipit-source-id: 5ffcfd5db99f30c730ede202c5e013afa682bac9
Summary:
Update the gitignore code so that patterns ending in a trailing slash correctly
match only directories. Fortunately we have file type information available
during TreeInode::computeDiff(), so getting the correct file type information
does not add any extra overhead. The older ignore checking code in
Dirstate.cpp can also do a reasonable job of getting file type information.
That code should also be removed at some point in the future, and use the
TreeInode logic for computing ignore status.
Reviewed By: bolinfest
Differential Revision: D4901326
fbshipit-source-id: 1222c8142876c91e1b80ec937ec84c0c28737224
Summary:
The temporary Hash object doesn't live long enough for the copy into
the string, so make an explicit temporary to extend its lifetime.
Reviewed By: simpkins
Differential Revision: D4900457
fbshipit-source-id: 948d919b929285c30ee1f9f0111cba20b60d4dc6
Summary:
When tree manifest data is available, we don't need to parse the full
manifest any more, which is great because we can simply and quickly load the
root manifest and be done with a checkout operation. We can then quickly load
the manifest entry for a given directory path on demand, making us closer to our
ideal of O(what-you-use).
This diff plumbs in the manifest code from mercurial so that we can reuse
the decoder implementation to get the manfiest data and translate it into
something that we can put into our LocalStore.
There's a little wrinkle when it comes to files; mercurial doesn't support
a means for getting the file contents based on just its hash, we have to
provide the filename and revision hash for that. We have existing code
to create proxy entries in the store to map from a proxy hash to a tuple
with the appropriate data needed to resolve the file entry.
We need to extend its use to the trees that we generate because we need
to be able to propagate that path down to a child tree when we generate
the tree entry for a file for the first time.
If a tree manifest is not available for a given revision, we fall back
to the existing implementation that parses the full flat manifest.
Note: the flat manifest import generates entries with different hashes
compared to the tree manifest hash version. While we could make the
serialization match up, there's little risk of this causing problems
for us in practice, so punting on that for now.
Reviewed By: simpkins
Differential Revision: D4873069
fbshipit-source-id: 865001996745700586f571ed67371aed1668a6a7
Summary:
This tweaks the main.cpp and EdenServer.cpp code so that we can avoid having
separate internal and open source builds of the eden/fs/service:server library.
Only one small function is different between the two builds. This updates the
code so that this function is only used from main.cpp, instead of being used
inside EdenServer.cpp.
Reviewed By: wez
Differential Revision: D4889721
fbshipit-source-id: 97abda92c36752516fb8a8e73b4cf50580db8d79
Summary:
This change makes it so that all of the C++ code related to the edenfs daemon
is now contained in the eden/fs subdirectory.
Reviewed By: bolinfest, wez
Differential Revision: D4889053
fbshipit-source-id: d0bd4774cc0bdb5d1d6b6f47d716ecae52391f37
Summary:
Move the code for the command-line tool up one directory, out of eden/fs.
This better separates the code so that eden/fs contains code for the edenfs
daemon, while eden/cli contains code for the command line tool.
Reviewed By: bolinfest
Differential Revision: D4888633
fbshipit-source-id: 5041e292c5353d05122eefe5db3257289e31239a
Summary:
Fix a subtle crash during checkout when handling newly added entries that
already exist in the working directory: CheckoutAction passed the entry name to
checkoutUpdateEntry() as a PathComponentPiece. However, this
PathComponentPiece could refer to the entry name owned by newScmEntry_, and it
also passed newScmEntry_ into checkoutUpdateEntry() as an rvalue reference.
As a result, if the string data was stored invalidated by the move the name
would no longer be valid when checkoutUpdateEntry() tried to use it.
This bug is triggered by doing an "hg update --clean", where a file added in
the destination commit already exists on disk, and has an entry name of 23
characters or less. (The 23 character limit is fbstring's upper bound on
small string optimizations, where it will store the string data inline in the
object, causing it to be invalidated on move.)
This also fixes a crash in a VLOG() statement when the verbose log level for
TreeInode.cpp was set to 4 or greater.
Reviewed By: bolinfest
Differential Revision: D4882544
fbshipit-source-id: 917ede6eeae2224aaa0724b8b30324f3c3a5c924
Summary:
This is a bit too magical -- it's not clear that the thing produces an Options object. If you do know that you can chain further option setters off this thing, it's nice, but otherwise, the first impression is "what just happened?".
So, let's have one good way for doing things.
Reviewed By: yfeldblum
Differential Revision: D4863947
fbshipit-source-id: 3dfe83cfc077d47f604f47dcb21149fbaa2d2243
Summary:
Update the hg extension to implement dirstate.rebuild(). This is necessary for
the `hg reset` command. This also now implements dirstate.setparents() for
cases when there is only one parent.
Reviewed By: wez
Differential Revision: D4823780
fbshipit-source-id: 802de006e03860995095dc3af17acb2eb05f4e8b
Summary:
Update the checkOutRevision() thrift API to accept the commit ID as a
BinaryHash rather than as a hexadecimal ASCII string, to make it more
consistent with our other thrift APIs.
Reviewed By: wez
Differential Revision: D4823779
fbshipit-source-id: 6226f926533ef86d60f8b1d659bc5d912e2855e6
Summary:
The intent is to provide a way to locate the SNAPSHOT file
for tools that want to have a very fast way to figure out the commit
id without making any RPCs or subprocess invocations.
Reviewed By: simpkins
Differential Revision: D4824176
fbshipit-source-id: 5adca225d9984146852dad1e83de0d903848c1e5
Summary:
This simplifies the InodePtrImpl class by removing support for pointer-to-const
Inode objects. We don't currently use pointer-to-const Inode objects, and I
don't really anticipate that we will ever need this.
The primary motivating factor behind this diff is a recent change in clang that
breaks the existing code: clang PR31606 prevents children classes from
inheriting template constructor instantiations from their parent which take a
reference to the parent class type as the only argument.
While this could have been fixed by explicitly defining the necessary
constructors in the child class (or simply by overriding newPtrLocked() and
newPtrFromExisting() to return the subclass type), dropping support for
pointer-to-const allows us to simplify the code and resolve this issue.
Reviewed By: wez
Differential Revision: D4825526
fbshipit-source-id: 999de352788b13818b7f23788bb34686e193cd5d
Summary:
previously, the importer would read the entire manifest
and emit data to the store as it resolved complete directory entries.
The entire manifest data would be buffered and sent out to the store.
In the scenario where one subtree has been modified and a commit has
been made, only the parents of the subdirectory need to be hashed
and stored, but we would compute and try to store everything anyway.
While this diff can't avoid having to compute hashes for everything (we need
tree manifest data for that), by breaking the import into two passes we can
potentially avoid interrogating the LocalStore about every tree in the entire
manifest during an import; we only need to store the trees that are missing and
can simply cut out sub trees that are already present.
This saves us IO at the expense of buffering the manifest tree in memory
for the duration of the first pass; it is an acceptable trade off.
Reviewed By: simpkins
Differential Revision: D4832166
fbshipit-source-id: 0a40cb851c65393b407a8161db05c4b1795fb11a
Summary:
This avoids hard coding knowledge of the FB-specific `local` dir.
This is only suitable for use on already-mounted eden instances,
so the function accepts either the mounted path or the eden
configuration dir.
Reviewed By: bolinfest
Differential Revision: D4829166
fbshipit-source-id: df9daed6024cdec6222be90f5cb624dd9ca0da0b
Summary:
Folly is dropping support for GCC 4.8 now that the branch for last HHVM release that needs to support it has been cut.
This codemod's away all the trivially identifiable uses of `folly::make_unique` in fbcode outside of folly, mcrouter and hphp.
A future diff will kill it in those, and a diff after that will kill it entirely.
Reviewed By: djwatson
Differential Revision: D4498157
fbshipit-source-id: 5a2c05ffd29e5358298acb53dab189a699b022cc
Summary:
This diff fixes FileData::stat() so that we report reasonable timestamp values
on non-materialized files, rather than always leaving them as 0. We set the
timestamps to the time that we created the FileInode. This ensures that
timestamps are updated correctly when files are modified by a checkout
operation.
Note that for materialized files the code reports the timestamp of the overlay
file. This diff does not modify that behavior. However, this behavior is
incorrect, as the overlay file timestamps are not updated by a FUSE client
opening, modifying, then closing a file (since we keep the underyling overlay
file handle open, and don't close it).
In the future we'll need to implement our own tracking of atime, mtime, and
ctime values. We should probably store these in a header inside the overlay
file. For now, this diff is a stop-gap measure that ensures we at least update
non-materialized file timestamps correctly on checkouts.
Reviewed By: bolinfest
Differential Revision: D4765632
fbshipit-source-id: 478da6441e213cdfe830f1c5129212182ce4eeb0
Summary:
This diff adds a new "eden debug" command, with a few subcommands:
- "eden debug inode": Report information about currently loaded inodes
- "eden debug tree": Report data about source control Tree objects
- "eden debug blobmeta": Report metadata about source control Blob objects
- "eden debug blob": Report the contents of source control Blob objects
This diff also includes the thrift APIs in edenfs to support these commands.
Reviewed By: bolinfest
Differential Revision: D4760528
fbshipit-source-id: 7dc2bd6e0e952ba6fb61702c672fb9417d603ffa
Summary:
Previously EdenServer::getMount() returned a null pointer if the path did not
refer to a known mount point. However, most of the thrift APIs using this
method never checked if the return value was null, which could allow any thrift
client to crash edenfs.
This updates getMount() to throw if the path is not found, and never return
null.
Reviewed By: bolinfest
Differential Revision: D4760527
fbshipit-source-id: 6cd9a1e3bc0a0a220d7f5fa518d497d37f245eff
Summary:
This updates all of the references to gtest and gmock with googletest.
The change is mechanilcal, generated with the following one-liner:
```lang=bash
hg grep -lwE '(gtest|gmock)' 'glob:**/TARGETS' | grep -v '^third-party-buck' | xargs perl -pi -e '
$gt=qr!(["'"'"'])gtest\g1!;
(
s!$gt(\s*,\s*(.any.|None))(\s*,\s*)?\),?!\1googletest\1\2, \1gtest\1\),!g or
s!$gt((\s*,\s*(.any.|None)[^\)]+))\),?!\1googletest\1\2\),!g or
s!\(\s*$gt,?\s*\),?!\(\1googletest\1, None, \1gtest\1\),!g or
s!$gt,?!\(\1googletest\1, None, \1gtest\1\),!g
) unless /(name|type) *=/;
$gm=qr!(["'"'"'])gmock\g1!;
(
s!$gm(\s*,\s*(.any.|None))(\s*,\s*)?\),?!\1googletest\1\2, \1gmock\1\),!g or
s!$gm((\s*,\s*(.any.|None)[^\)]+))\),?!\1googletest\1\2\),!g or
s!\(\s*$gm,?\s*\),?!\(\1googletest\1, None, \1gmock\1\),!g or
s!$gm,?!\(\1googletest\1, None, \1gmock\1\),!g
) unless /(name|type) *=/;
'
```
Reviewed By: meyering
Differential Revision: D4643237
fbshipit-source-id: fda7f41760c7e44254231df87634631c343e6355
Summary:
The hg_import_helper script that eden uses to import data from mercurial keeps
a long-lived repository object open. This caches some data about the
repository, and if new commits are added after it was created, it can fail to
see them.
This updates hg_import_helper.py to catch errors that occur when trying to use
the repository objects. The code will invalidate the repository object and
then retry the operation once, in the hopes that it will now succeed after
invalidation.
Reviewed By: bolinfest
Differential Revision: D4752659
fbshipit-source-id: 1c75c84766d6bbda0710882a338eaa09e0cb0030
Summary:
Tweaks the stats stuff so that we can name the histograms and export them with p50, p90, p99 percentiles.
This is made a bit ugly because the fb303 stats code isn't yet open source, so
there's a moderately ugly preprocessor directive that we assume to be true for
our internal build. We'll need to force that to be 0 for the open source build
until we open the stats code and make it available.
Reviewed By: bolinfest
Differential Revision: D4801868
fbshipit-source-id: 643909e63bd4a74b2cfa580be131f65c5673bc94
Summary:
This explicitly uses a WriteBatch during importing. This results in a
5x perf gain for import operations.
I've also added in a read-before-write check to make sure that we're not
attempting to overwrite a key that already exists. Since this is a
content-addressed store, a given key should always have the same
value; it should be immutable. Overwriting an existing key is an
opportunity to change data that should never change and is also wasteful
of IO bandwidth.
Ideally we'd be able to exploit this to avoid descending into a directory;
I'd like to hook that up to the treemanifest stuff in mercurial in a
follow on diff.
Reviewed By: bolinfest
Differential Revision: D4783598
fbshipit-source-id: 24f95495d09a852859b346559fdf83d064de2b51
Summary:
this is the bare minimum to support creating unix domain sockets.
We only support using mknod to create a unix socket; other uses will yield an error.
I've added an rdev field as a sibling of the existing mode field that we track,
as that is the additional parameter that we need to track as part of the
special file node.
Special file nodes are tracked in the overlay as empty files.
Reviewed By: bolinfest
Differential Revision: D4774099
fbshipit-source-id: 0824b7e509063faa8bede7aff82a7c51930c4f83
Summary:
This is an initial pass at implementing "hg status" using the new EdenMount and
TreeInode diff() code.
More work still needs to be done to clean things up here, but this makes
"hg status" more correct in the face of modified files and directories that are
not modified, and vice-versa.
In particular, the following still needs to be cleaned up in future diffs:
- Update the "hg status" code to more correctly process user directives for
paths that did not appear in the diff output. This will probably be
simplified some by correctly updating the user directive list on checkout and
resetCommit() operations. It may also make things simpler if we store the
user directives in a hierarchical path map, rather than in a flat list.
- Update the addAll() code to also use the new diff logic, rather than the
getStatusForExistingDirectory() function.
- Clean up the locking some. I think it may be worth changing the code to use
a single lock to manage both the current commit ID for the mount and the
dirstate user directives. We probably should hold this lock for the duration
of a diff operation. The diff operation should also return the commit ID
that it is based against.
Reviewed By: wez
Differential Revision: D4752281
fbshipit-source-id: 6a6d7e2815321b09c75e95aa7460b0da41931dc0
Summary:
It's not really magic because we don't have a virtual directory
inode base any more. Instead, we mkdir and populate it at mount time.
What is slightly magical about it is that we give it some special powers:
* We know the inode number of the eden dir and prevent unlink operations
on it or inside it.
* The .eden dir is present in the contents of the root inode and will
show up when that directory is `readdir`'d
* When resolving a child of a TreeInode by name, we know to return the
magic `.eden` inode number. This means that it is possible to `stat`
and consume the `.eden` directory from any directory inside the eden
mount, even though it won't show up in `readdir` for those child dirs.
The contents of the `.eden` dir are:
* `socket` - a symlink back to the unix domain socket that our thrift
server is listening on. This means that it is a simple
`readlink(".eden/socket")` operation to discover both whether a directory
is part of an eden mount and how to talk to the server.
* `root` - a symlink back to the root of this eden mount. This allows
using `readlink(".eden/root")` as a simple 1-step operation to find
the root of an eden mount, and avoids needing to walk up directory
by directory as is the common pattern for locating `.hg` or `.git`
dirs.
Reviewed By: simpkins
Differential Revision: D4637285
fbshipit-source-id: 0eabf98b29144acccef5c83bd367493399dc55bb
Summary:
Update the EdenMount::diff() code to completely skip .hg and .eden directories.
This was implemented through the GitIgnore code, and adding a new HIDDEN status
The .hg and .eden directories are similar to ignored directories, but we never
want to report any information for them even when listIgnored is true.
This also changes the GitIgnore code to so that GitIgnoreStack computes the
basename for each path once, and passes it down into the underlying match code,
so we don't have to re-compute it for each pattern that needs the basename.
Reviewed By: wez
Differential Revision: D4751917
fbshipit-source-id: ec22d62f31a3ce9ef998a8ccb4413f72f1d7a487
Summary:
This updates the diff() logic to process gitignore rules.
The bulk of the existing TreeInode::diff() was moved into a
TreeInode::computeDiff() function. TreeInode::diff() now loads the gitignore
rules for this directory before calling computeDiff(). The argument list for
computeDiff() was now getting slightly unwieldy, so I created a new DiffContext
class to hold some of the arguments that are the same for the duration of the
entire diff operation. This resulted in some rather mechanical changes through
the computeDiff() and DeferredDiffEntry code to update the argument lists.
Reviewed By: wez
Differential Revision: D4744690
fbshipit-source-id: c4981d9f49878b522e0b3faf35de837739066f3c
Summary:
This tweaks the definition of the subscribe method in the
streamingeden thrift file and implements the server side of the
thrift service, and the client side to consume it in watchman.
The definition is now a bit simpler than it was previously; we're
now just sending a stream of the updated JournalPosition objects
to the client rather than computing and sending FileDelta objects.
This is cheaper for Eden to deal with because it is very cheap to take
the current journal position and pass that over the wire. This is
important because a burst of mutations will likely queue up a bunch
of these in quick succession.
In a future diff I'm considering adding a latency parameter for
the subscription so that we can constrain the number of updates
over a certain time period (likely in the 10's of milliseconds range).
For now I just want to prove that the concept works.
On the watchman side we just need to pull these off the wire as they are sent
by eden, then wait for the subscription stream to settle before internally
broadcasting to any connected subscribers.
Reviewed By: bolinfest
Differential Revision: D4647259
fbshipit-source-id: 03aa16e59a43195a2505a8d03bce1ccf88a8d42f
Summary:
Update the edenfs main function to check the exit code of the privhelper
process, and to exit with EX_SOFTWARE rather than EX_OK if the privhelper did
not exit with status 0.
This will ensure that our integration tests always catch crashes in the
privhelper process. Previously we could catch errors that happened before the
privhelper socket was closed, but if the privhelper server crashed during
cleanup we never noticed the error.
Reviewed By: bolinfest
Differential Revision: D4728006
fbshipit-source-id: 2c478612e5fabec90438b28b9c76a3343f561655
Summary:
Add a unit test that modifies a file and then does a forced checkout to revert
back to the original commit state.
This also includes some pretty-printers to make the tests print
CheckoutConflict objects nicely if any test assertions fail.
Reviewed By: bolinfest
Differential Revision: D4727779
fbshipit-source-id: b31dc90be28d17ed0f2a4076c96a08ba82588190
Summary:
This begins adding an EdenMount::diff() method, which walks through the inode
tree and reports differences from the current source control tree state.
My intent is to eventually update the Dirstate code to use this diff()
function, and to remove the existing getModifiedDirectories() logic. The
getModifiedDirectories() code is currently incorrect, as it reports
materialized directories, which is not the same as directories that are
modified from the current source control state.
The ignore processing logic is not currently implemented in EdenMount::diff(),
but it should be relatively straightforward to add in a subsequent diff.
Reviewed By: bolinfest
Differential Revision: D4726048
fbshipit-source-id: ad0bb3b5d72bb3830f60fc2b2e56a81217c35353
Summary:
Update the TreeInode and CheckoutAction to correctly be able to remove
directories during checkout operations. This handles both removing directories
that are newly empty as a result of the checkout, and directories that are
replaced with a file or symlink.
This also refactors the rmdir/unlink code a bit so we can share some of its
logic for doing directory removal during the checkout operation.
Reviewed By: bolinfest
Differential Revision: D4708607
fbshipit-source-id: 3b9dd9cc3536845dad0d9278e3c5d3ac1ed04570
Summary:
Synchronize access to the CheckoutContext conflicts_ list when recording
conflicts or errors during a checkout operation.
In theory it is possible for the checkout operation to be proceeding
simultaneously in multiple threads. At the moment this does not happen because
the BackingStore implementations currently block, and always return
immediately-ready Future objects in the same thread. However, once they are
updated to return Future objects that actually complete asynchronously in other
threads the checkout work will continue in these other threads.
Reviewed By: bolinfest
Differential Revision: D4708605
fbshipit-source-id: 885b0114d97dadcacf64652f795a7d3846412f11
Summary:
Add a test that exercises a checkout operation that creates a brand new
subdirectory.
Reviewed By: bolinfest
Differential Revision: D4693994
fbshipit-source-id: 70f43f67c2cdb10bd854f51d5ba2e61c45976975
Summary:
Unfortunately it looks like gcc won't support the [[nodiscard]] attribute until
the gcc-7.x release. (The functionality is available in gcc-4.8 and later, but
using the name [[gnu::warn_unused_result]] instead.)
This changes the code to use the FOLLY_WARN_UNUSED_RESULT macro from
folly/Portability.h instead. (This expands to older non-standard __attribute__
syntax for gcc and clang, and _Check_return_ on Windows.)
Reviewed By: bolinfest
Differential Revision: D4704184
fbshipit-source-id: d5c13630ea611a2f43080a501add42ce9fda6789
Summary:
Add new ASSIGN_PARENT_INODE and ASSIGN_INODE types to LoadBehavior, to exercise
additional code paths during checkout operations. This also cleans up the
overall handling of the LoadBehavior enum.
Reviewed By: bolinfest
Differential Revision: D4694548
fbshipit-source-id: e8359cfdadb1fabb1a0d07bc57a1a1610ed9ba57
Summary:
Refactor FileInode and FileData so that they no longer store the
TreeInode::Entry that refers to this file. The TreeInode::Entry is owned by
the parent TreeInode and is not safe to access without holding the TreeInode
contents_ lock, which the FileInode code never did. Additionally, once a
FileInode is unlocked the TreeInode::Entry is destroyed. Previously the
FileInode code would retain its pointer to this now invalid data, and could
still dereference it.
In the future I think it might be worth refactoring the FileInode/FileData
split a bit more, but this at least addresses the issues around invalid
accesses to entry_.
Reviewed By: bolinfest
Differential Revision: D4688058
fbshipit-source-id: 17d5d0c4e756afc6e3c4bb279cb1bb5231f3134f
Summary:
Remove the TreeInode::entry_ member variable. This was a pointer to the
TreeInode::Entry object representing this inode in the parent TreeInode.
However, it isn't safe for a child Inode to access the TreeInode::Entry owned
by its parent. This Entry object is supposed to be protected by the parent
TreeInode's contents_ lock, and the child inode never held this when accessing
the Entry.
This updates the TreeInode code to never use entry_ any more. I will work on
removing FileInode::entry_ in a subsequent diff.
This did require changing the overlay materialization code some. Previously
the checkout code directly modified entry_ to inform the parent TreeInode if
the child was materialized or not. Now we have to more explicitly tell the
parent TreeInode about changes to each child's materialization state. With
this change we now keep the overlay data on disk more consistent during the
checkout process. We should now always have overlay data for children inodes
on disk whenever the parent thinks they are materialized. We may end up
writing out TreeInode overlay data frequently during the checkout process,
however. We might be able to improve this in the future. For now it seemed
simpler and safer to just update the data frequently to make sure it stays
consistent.
Reviewed By: bolinfest
Differential Revision: D4688055
fbshipit-source-id: d08a14b9304ac49826f0897a0b53281177d9d9f4
Summary:
Update the code so that we no longer immediately remove overlay data about a
file or directory when the inode is unlinked. The file can continue being used
after it was unlinked if there are open file handles or other internal
references to it.
This updates the code to instead wait to remove the overlay data until the
InodeBase object is unloaded.
This does not fully fix all the outstanding issues with using inodes after they
have been unlinked, because the FileInode and TreeInode still use the entry_
pointer that now points to a bogus TreeInode::Entry.
Reviewed By: bolinfest
Differential Revision: D4688056
fbshipit-source-id: a9e1c1a9562590b8b81237057bfc4ef77795807f
Summary:
This changes a couple aspects of the materialization process:
- Write out child files in the overlay before their parent. If we throw an
exception or crash partway through this process, it is better to have the
child not marked materialized in its parent, rather than to have the parent
indicate that the child is materialized but to not actually have any overlay
data for the child. In the former case we will still load the correct child
data the next time we need to, but in the latter case we would fail to load
the child correctly.
- Hold the rename lock while materializing our parent directories, to ensure
that we materialize the correct parent, and it cannot change while we are
trying to perform the materialization.
- Make the parent responsible for modifying the child's TreeInode::Entry. This
data is owned by the parent and is protected by the parent's contents_ lock.
The child really shouldn't ever be directly accessing this data without
holding the parent's contents_ lock.
Reviewed By: bolinfest
Differential Revision: D4688057
fbshipit-source-id: 2662f79cb7d7febb086f4e0888a3d96a580c4bfa
Summary:
This fixes the checkout code to always recurse into directories with conflicts,
and accurately list all individual files with conflicts.
Previously the checkout code always avoided recursing into unloaded directory
inodes that were affected by the checkout. Since the inode was not loaded we
can easily replace it without recursing into it. Unfortunately this resulted
in an inaccurate conflict list--we would report the directory itself as a
conflict. This is insufficient for our mercurial extension, which needs to
know the specific list of files with conflicts, so it can ask mercurial to
perform conflict resolution on them.
Reviewed By: bolinfest
Differential Revision: D4672373
fbshipit-source-id: ff400a4a6e25b8b0754c66cb067a69d553a5c98b
Summary:
Loading the child inode is necessary to avoid race conditions with other
threads that may be currently trying to load the child inode in question.
This updates the code so that rmdir() and unlink() now share most of their
implementation.
Reviewed By: bolinfest
Differential Revision: D4664110
fbshipit-source-id: 871ca7b3c22819b24bc2639d9550bb45f6541f50
Summary:
Update the DEFS and .buckconfig files so that the C++ code can be built
successfully with buck in our github repository.
Reviewed By: wez
Differential Revision: D4633749
fbshipit-source-id: a07a6e664471237fcc2d13a8be8b92efa3a101bf
Summary:
Replace the one very basic checkout unit test with several tests that
more thoroughly test file modification changes. More tests for
directory changes and file type changes still need to be added, but this is a
first step for now. I will continue adding more tests in upcoming diffs.
Reviewed By: wez
Differential Revision: D4641595
fbshipit-source-id: 443a68e4e3c673fe687446f3469f035b4f48f1b3
Summary:
Add a method to reset the current commit without changing the working
directory state, similar to "hg reset" and "git reset --soft".
This also adds a new EXPECT_FILE_INODE() check for use in the unit tests,
and FileInode::getPermissions() and FileData::readFull() methods to support
this check.
Reviewed By: wez
Differential Revision: D4641476
fbshipit-source-id: 1e516774fe8e292a8d82cc2c354619374a3abe37
Summary:
The FakeTreeBuilder class in D4609587 provides a lot of the same functionality
as the TestMountBuilder class, but is not restricted to being used only at
mount initialization time. (It also provides more powerful functionality for
controlling the ordering of events when loading data from the ObjectStore.)
This switches all of the existing tests to use FakeTreeBuilder rather than
TestMountBuilder, and removes the TestMountBuilder class.
One difference is that FakeTreeBuilder adds data to the FakeBackingStore, while
TestMountBuilder previously put data directly into the LocalStore. This
shouldn't really make much difference for the behavior of the code. Putting
data in the FakeBackingStore gives us more control over when data is available
during load operations, and exercises slightly more of the normal ObjectStore
code path.
Reviewed By: wez
Differential Revision: D4641288
fbshipit-source-id: ca2b45bf69bd373848c12a7b739b286aabe6aa9b
Summary:
Add a FakeTreeBuilder class to make it easier to create Tree structures in the
FakeBackingStore using path names. Previously the test code had to manually
chain together Trees and Blobs. FakeTreeBuilder lets callers call setFile()
with a path name to define the file layout. The files that were defined then
get converted into Tree and Blob data when finalize() is called.
This is similar to the existing TestMountBuilder::addFile() API. However,
TestMountBuilder only populates the LocalStore. FakeTreeBuilder populates
the FakeBackingStore instead. The FakeBackingStore class allows tests to have
much more control about when objects appear ready from the backing store,
allowing control over the order in which Futures are fulfilled during the test.
Reviewed By: bolinfest
Differential Revision: D4609587
fbshipit-source-id: 9642d17daf0dc10f08901e51552bbb3c3e150b53
Summary:
Make sure the eden coded compiles cleanly with -Wshadow-compatible-local
Pretty much all of the warnings were issues with lambdas shadowing names from
their parent context (even though they didn't ask to capture those names from
the parent).
Reviewed By: wez
Differential Revision: D4644849
fbshipit-source-id: 66629cd98b5af4760f3fbb256e44c0bc47e52316
Summary:
Previously, we would only allow you to consume a symlink that was
checked into a revision, but not allow you to create a new one.
This diff adds support for making new symlinks.
It turns out that the code that we had for readlink had an unused code path for
reading an actual symlink out of the overlay. My first pass at this diff tried
to make the materialization code generate such a symlink, but it was breakin
some other assumptions in the rest of the code.
This version of the code just stores the symlink target in the file contents.
This means that we can simplify readlink to just calling `readAll` on the
underlying file data, and that will load the contents out of the storage layer
if the file wasn't materialized. This helps simplify things a bit more.
Reviewed By: bolinfest, simpkins
Differential Revision: D4604016
fbshipit-source-id: 3138d0f9880639d2bbeaf4bb03bef3f021c3ecb3
Summary:
In order to send realtime subscription information to watchman
we need to use the thrift streaming data functionality. Since this
isn't supported in java we originally broke this out into its own
thrift file.
It's now time to hook this up; this diff adds some plumbing so that
our thrift server knows to implement this interface.
This was made a bit more complicated because the inheritance in
the thrift definitions causes some ambiguities in the header maps
that buck uses to manage the includes.
I also had to rename the file from `eden.stream.thrift` to `streamingeden.thrift`
to avoid a problem with the period in the middle of the identifier.
For the server side of things, I've removed the `:thrift-cpp2` target
and replaced it with `thrift-streaming` instead.
This diff doesn't implement the endpoint, it is just setting things
up for the follow-on diff that does.
Reviewed By: bolinfest
Differential Revision: D4640639
fbshipit-source-id: b3625b0ff33a42945bf523944beed050f549a18c
Summary:
I originally added this to facilitate `hg status` but we didn't end up
using it. Rather than keeping it around and updating the tests in the `.eden`
dir diffs that follow, let's just remove it.
Reviewed By: simpkins
Differential Revision: D4610768
fbshipit-source-id: 158c0207f88980e86aeeddf75e6fd49763d2a402
Summary:
Remove AutoHeaders.RECURSIVE_GLOB from all of our TARGETS files. This is a
Facebook-internal directive that was added as part of a codemod. The vanilla
open source buck does not know about this directive, and can't parse TARGETS
files that contain it.
All of the affected rules are cpp_unittest() and cpp_binary() rules that don't
actually have any header files, so these parameters didn't actually have any
effect.
Reviewed By: andrewjcg
Differential Revision: D4633748
fbshipit-source-id: 6227fe9b7f2ea838a0c72408ca195685bcae9f5a
Summary:
In TreeInode::computeCheckoutActions() the code incremented the wrong index
when we reached the end of the new tree before the old tree.
panda-facepalm
Reviewed By: bolinfest
Differential Revision: D4609906
fbshipit-source-id: 61ca6baabe4eebb033c48b742562558466a45104
Summary:
Invalidate directory entries as they are modified during the checkout
operation. We never need to invalidate inode contents, since we always update
files by replacing the file with a new inode.
Reviewed By: wez
Differential Revision: D4577942
fbshipit-source-id: afde3040960b076d2cd384301a89cd7bdfce5a6a
Summary:
When the "eden daemon" CLI command has to run edenfs via sudo, also explicitly
pass through the KRB5CCNAME environment variable. This will allow edenfs and
it's hg import helper to find the correct Kerberos credentials if it needs to
connect to a remote server.
Reviewed By: wez, bolinfest
Differential Revision: D4584132
fbshipit-source-id: 6c72e4eca76d81b3d4b53a9e72a0e8388da30a7d
Summary:
This updates CheckoutAction to make the old TreeEntry object optional.
This is necessary to support situations where the entry did not exist in the
old source control tree, but does exist in the new tree and also exists locally
on the file system. This situation is always a conflict, so it only applies
when a force checkout is being done.
This diff also refactors the TreeInode::computeCheckoutActions() logic to
combine the processNewEntry/processRemovedEntry/processEntry lambdas into a
single function, as suggested in the code review for D4538516.
Reviewed By: wez
Differential Revision: D4577924
fbshipit-source-id: 9435205b541a12c75a8b9bd2b2b599a339a2f6c8
Summary:
Add a thrift call to get the ID of the snapshot currently checked out in a
given eden mount.
Reviewed By: bolinfest
Differential Revision: D4570729
fbshipit-source-id: 208ceb309aacfb07e30ddb8fdfc0950a3d7d748b
Summary:
Fix the FileHandle code that was calling getPathBuggy() when writing journal
entries. This fixes a crash when writing to an unlinked file.
Also remove an unused call to getPathBuggy() in TreeInodeDirHandle.
These were the last places calling getPathBuggy(), so I have now deleted this
method entirely.
Reviewed By: bolinfest
Differential Revision: D4570606
fbshipit-source-id: a1c4aab10df96e2aeee78fca6d5808db28ad0cbf
Summary:
This is the initial code for implementing checkout.
This isn't quite 100% implemented yet, but I think it's worth checking in this
code as-is, and getting the remaining functionality done in separate diffs.
In particular, a few operations aren't implemented:
- Removing a directory that was deleted in the new revision
- Replacing a directory that was replaced with a file or symlink in the new
revision
- When doing a forced update, replacing a file or directory that did not exist
in the old revision, but that was created locally in the working directory,
and also exists in the new revision.
Reviewed By: wez
Differential Revision: D4538516
fbshipit-source-id: 5bb4889b02f23ab2048fcae2c8b7614340181aa6
Summary:
Update PrivHelperServer::bindUnmount() to give up after 2 seconds instead of
possibly looping forever.
Also fix the unmount code in the eden process to rethrow the original exception
as-is. Previously it was re-throwing it as a std::exception, losing the
exception details.
I think there are still other bugs here in this unmount code--it repeatedly
hangs for me on unmount. (I believe it is trying to unmount things that aren't
actually bind mounts.) However, I'll look into fixing those in a separate
diff.
Reviewed By: wez
Differential Revision: D4547711
fbshipit-source-id: af6bce0262e8314361450874ef998ebf7ad590e3
Summary:
Refactor the Overlay code to store data using inode numbers rather than the
affected file's path in the repository. This simplifies the TreeInode code a
bit, as we no longer have to rename overlay files to stay in sync with the file
paths. This also eliminates some crashes when trying to update overlay files
for inodes that have been unlinked (and hence no longer have a path). This
also includes a few fixes to avoid writing journal entries for unlinked files
too. Additionally this contains a few fixes to how mode bits are stored in the
overlay, and fixes a bug where create() was ignoring the mode argument.
Reviewed By: wez
Differential Revision: D4517578
fbshipit-source-id: c1e31497dcf62c322b0deff72b0a02675b0509ab
Summary:
InodeBase::getNodeId() already returns the inode number, so there is no need
for the TreeInode subclass to provide an alternatively-named method that does
the same thing.
Reviewed By: bolinfest
Differential Revision: D4500215
fbshipit-source-id: fbc595cf9e8f43efe44ed352a97e3a4974b5aed0
Summary:
It turns out that the manifest class provides an iterentries() method to
iterate through the (path, hash, flags) values. Update the code to use this
rather than iterating through the paths and then having to look up the hash and
flags using the path.
This dramatically speeds up importing a commit with 1M files: the import time
drops from around 8 seconds to under 1.5 seconds. (Of the 1.5 seconds, about
half is spent reading the manifest data, and half is spent formatting and
writing the chunks.)
Reviewed By: wez
Differential Revision: D4512538
fbshipit-source-id: ccfff162082d327d525224b2d7dc87f67bda5e22
Summary:
This updates the TreeInode::rename() code to handle concurrency better.
In particular:
- The code now ensures that both the source inode being renamed and destination
inode (if it exists) are loaded. This simplifies issues when an inode is
being loaded at the same time a rename is in progress. This ensures that any
pending load is processed before the rename takes place. (All promises for
the load might not be fulfilled before the rename completes, but the relevant
TreeInode and InodeMap data structures are updated before the rename occurs.)
This does mean that the rename code potentially might have to retry several
times if the inode it began loading is no longer the affected source or
destination or child once the load completes. However, this seems like a
reasonable trade-off, compared to dealing with the complications that would
arise with the load code having to handle renames occuring before load
completion.
- The code now implements proper lock ordering, to avoid acquiring locks in
conflicting orders that might cause deadlock with other threads also trying
to acquire the same locks. The InodeLocks.md document has been updated to
clarify the TreeInode lock ordering requirements.
Reviewed By: wez
Differential Revision: D4493526
fbshipit-source-id: 627393fafad90eb551aea62be7762d59ed043abe
Summary:
Move the integration tests from eden/fs/integration up one directory, to
eden/integration.
The main benefit is that this makes it easy to run just the edenfs unit tests
by running "buck test eden/fs/...". These unit tests complete much more
quickly than the full set of integration tests, providing a faster test suite
to re-run repeatedly during development. The integration tests can be run with
"buck test eden/integration/...", and the full set of tests can still be run
with "buck test eden/..."
Reviewed By: wez
Differential Revision: D4490247
fbshipit-source-id: 5ceb5a19526f56e1cb926f352fa30ad2f1212c05
Summary:
Most of the FileData logic assumes that file_ is open if the file is
materialized. Fix the FileData() constructor to actually open file_ if the
entry is marked materialized, so that this assumption is actually true.
Reviewed By: wez
Differential Revision: D4471365
fbshipit-source-id: b399e6e5e25eaf12f4204f5835db01baa5a6cd78
Summary:
Update the code to release the parent TreeInode's contents lock when processing
its children. This makes sure we do not hold the root lock for the entire
duration of this function.
Reviewed By: wez
Differential Revision: D4471289
fbshipit-source-id: dc4b50bd06fcceb7a7df3d7dd04cfc0d41285556
Summary:
This is to facilitate the watchman integration and draws on the
watchman glob implementation; the approach is to split the glob strings into
path components and evaluate the components step by step as the tree is walked.
Components that do not include any glob special characters can be handled as
a direct lookup from the directory contents (O(1) rather than O(num-entries)).
The glob method returns a set of filenames that match a list of
of glob patterns.
Recursive globs are supported. It is worth noting that a glob like "**/*" will
return a list of every entry in the filesystem. This is potentially expensive
and should be avoided. simpkins is in favor of disallowing this as a forcing
function to encourage tool-makers to adopt patterns that don't rely on a
complete listing of the filesystem.
For now I'd like to get this in without such a restriction; it's also worth
noting that running `find .` in the root of the mount point has a similar
effect and we can't prevent that from happening, so the effect of the overly
broad glob is something that we need to be able to withstand in any case.
Unrestricted recursive globs will make it easier to connect certain watchman
queries in the interim, until we have a more expressive thrift API for walking
and filtering the list of files.
Note: I've removed the wildmatch flags that I'd put in the API when I stubbed
it out originally. Since this is built on top of our GlobMatcher code and that
doesn't have those flags, I thought it would be simplest to just remove them.
If we find that we need them, we can figure out how to add them later.
Also Note: the evaluation of the glob is parallel-ready but currently limited
to 1 at a time by constraining the folly::window call to 1. We could make this
larger but would need a more intelligent constraint. For example, a recursive
glob could initiate N concurrent futures per level where N is the number of
sub-dirs at a given level. Using a custom Executor for these futures may be a
better option to set an upper bound on the number of concurrent jobs allowed
for a given glob call.
Depends on D4361197
Reviewed By: simpkins
Differential Revision: D4371934
fbshipit-source-id: 444735600bc16d2c2185f2277ddc5b51f672600a
Summary:
As part of the mount process, make sure we load InodeBase objects for all files
in the mount that are already materialized.
Other parts of the code assume that InodeBase objects are always loaded for
materialized files. We never unload materialized inodes, but if the mount
point was unmounted then remounted we previously did not ensure to load the
materialized InodeBase objects. This diff makes sure we load all materialized
inodes before starting the mount.
Reviewed By: bolinfest
Differential Revision: D4461193
fbshipit-source-id: 70d06fd01e2df333ce2816d5d7a392b0a5d6e1e6
Summary:
Fix two issues with inode load completion:
1) Make sure we release the parent's TreeInode lock before fulfilling promises
waiting on the inode load to complete. Otherwise this can result in deadlock
if the promises have callbacks that then try to acquire the TreeInode lock.
2) Use unique_ptr<InodeBase> in the load completion code before we have
inserted the Inode into the InodeMap, rather than using InodePtr. The new
InodePtr class relies on the InodeMap to handle ownership, and it cannot
destroy Inode objects unless they are in the InodeMap. Using a unique_ptr
instead ensures that the InodeBase will be destroyed properly if an error
occurs between when we construct the InodeBase and when we insert it into the
map.
Reviewed By: bolinfest
Differential Revision: D4427889
fbshipit-source-id: 2627a4956735f4acebe5c259abd8de0430ff6177
Summary:
Rename the current EdenMount::getInodeBase() function to
EdenMount::getInodeBaseBlocking() and add a new getInodeBase() function that
performs the lookup asynchronously and returns a Future<InodePtr>.
This is implemented with a TreeInode::getChildRecursive() function that allows
asynchronous recursive lookups at any point in the tree.
Reviewed By: bolinfest
Differential Revision: D4427855
fbshipit-source-id: aca55e681a48c848b085b7fc6a13efe6cf0a4e3a
Summary:
Add methods to InodeBase to get the parent of the current Inode, and remove the
existing TreeInode::getParent() function. The old TreeInode parent_ member was
never updated properly after rename or unlink operations, so it could return
incorrect information.
Reviewed By: bolinfest
Differential Revision: D4427103
fbshipit-source-id: cb66a014f745b4af31033e0a3c5405e29791b869
Summary:
Thesis: Since we don't yet have `hg update` implemented, these attributes can be cached forever.
When we implement `hg update` we will explicitly inform the kernel to invalidate anyway.
... and it turns out that the `scmRemove` thrift call is making changes that
aren't visible to the kernel already so we need to hook up some basic
invalidation.
This diff includes the bare minimum code to facilitate this; I've added a
helper method to the fusell::RequestData class to test whether we are in the
context of a FUSE request.
In the EdenDispatcher::unlink method we need to explicitly invalidate if we are
not being called via FUSE.
I have mixed feelings about putting this code in here. Given the initial
mental model I had in the prototype, this is the right place to put it, but it
does mean that all tree mutations must have via the Dispatcher in order for
invalidations to be routed to the correct places. Do we want to move away from
this?
It wouldn't be the end of the world to expand this diff a bit to add
similar/appropriate invalidation calls to all the mutation methods in our
dispatcher implementations while we figure out if we want to keep this.
Reviewed By: simpkins
Differential Revision: D4452961
fbshipit-source-id: 9471f145242fce0620c6872b74b02c56c8d78af1
Summary:
This mirrors the capability added in D4444058.
I don't think the `getenv` calls are awesome, but also don't think that we're
likely to want to override these outside of the environment either, so it
doesn't warrant plumbing these all the way through the eden cli, edenfs command
line flags and the layers to get to instantiate this object.
Reviewed By: simpkins
Differential Revision: D4446320
fbshipit-source-id: d5661e4f3e8dee82617eb6edddbcb9da5f4296d2
Summary:
While testing with the fb-eden rpm installed, I hit some integration
test failures. These were caused by the integration tests picking up the
default post-clone hook configuration.
This diff changes our existing `systemConfigDir` option (which defaults to
`/etc/eden/config.d`) to `etcEdenDir` (which defaults to `/etc/eden`) and
adjusts the code that consumed `systemConfigDir` to construct the effective
value by computing `etcEdenDir + "config.d"`.
Doing this allows us to also default the `repoHooks` path to be
`etcEdenDir + "hooks"` rather than just hard coding `/etc/eden/hooks`.
The result of this is that our integration tests will now pass when `fb-eden`
is installed, because they override the `etcEdenDir` option and isolate their
`edenfs` processes from the globally installed configuration.
Reviewed By: bolinfest
Differential Revision: D4446321
fbshipit-source-id: 524fdb2f386fdf16dce42dce7661d07e13c0f0e7
Summary:
It's been bothering me that we claim to run and complete running
the globally installed hooks when they aren't there, so tidy up the logging.
Reviewed By: bolinfest
Differential Revision: D4446318
fbshipit-source-id: acda223e1df2302602ad9967b1aa1b334b8477c7
Summary:
This adopts the new InterpolatedPropertyTree class as
the configuration storage.
It doesn't introduce any actual configuration of the interpolation
replacements; that will be in a follow-on diff.
Reviewed By: bolinfest
Differential Revision: D4444157
fbshipit-source-id: 18ead8e9074d23b1154e81f012f0c90efced1350
Summary:
This makes it possible to deploy via packaging (we'll put the par file
in the same dir as the extension) and eliminates a recursive buck build call
that was slowing down our integration tests; we can now just look up the path
from the environment that we set up in the TARGETS file to locate the par files
in the integration tests.
Reviewed By: bolinfest
Differential Revision: D4444096
fbshipit-source-id: 2ad6b6aa783563dc54a1cfd347a509c9919bb17a
Summary:
I don't know that I'm happy with the name; it started out as
just a helper wrapper to apply interpolation, but grew a bit to better
encapsulate how we access the config from the eden server.
This is a utility class that I'd like to use in place of the raw boost
property tree in our ClientConfig code.
It encapsulates the underlying property tree and provides a couple of simple
methods; one for loading data from a config file and one for looking up
a configuration value.
The configuration value lookup will replace tokens of the form `${KEY}`
with the value of `KEY` from its internal replacements dictionary.
This mirrors the interpolation handling that we do in the equivalent python
code for the eden client.
Reviewed By: bolinfest
Differential Revision: D4444075
fbshipit-source-id: 5d46d63f87caad4f409fbb981aa83165fcd6596d
Summary:
I'm going to follow up with some changes to substitute
some core env vars; this is just some prep.
Reviewed By: bolinfest
Differential Revision: D4444030
fbshipit-source-id: ab5d0e39aba14cdba39bb4867ec24955665723c6
Summary:
`eden config` dumps all parsed config values.
`eden config --get SECTION.OPTION` prints the value of the corresponding key.
Reviewed By: bolinfest
Differential Revision: D4434084
fbshipit-source-id: 63d9f72aa8794371e89a6d9e527bacaab17540de
Summary:
This makes it easier to reason about where these things can be found,
and in particular, allows passing the path through to the post-clone hook
script. Although post-clone accepts some command line configuration options,
our hooks scheme doesn't seem to allow setting that in the eden configuration
file.
Reviewed By: bolinfest
Differential Revision: D4443872
fbshipit-source-id: ed92dbac3b91b7f7dc86a39a85b465907f290b47
Summary:
We implement some sanity checking in wrappers, preventing some unsafe
consequences of using raw Buck UI in fbcode.
Reviewed By: Gownta
Differential Revision: D4453147
fbshipit-source-id: 3069898069c7e89223b133224f7c87d1a6b5886a
Summary:
Fix issues flagged by running "arc lint" on all eden files. There are still 6
warnings outstanding, but these are either false positives or advice that we
intentionally are ignoring for legitimate reasons.
Reviewed By: bolinfest
Differential Revision: D4446615
fbshipit-source-id: 992f3c146f99d63935f849aa775dd6d611a04acf
Summary:
10 seconds isn't long enough for the tests to pass consistently on my devserver (I have a windows VM running on here, so there is more background crap running).
Raising this up to 30 gave me a successful test run.
Reviewed By: bolinfest
Differential Revision: D4442300
fbshipit-source-id: 5c3ae1fc0d0acb21c5bb44e36dae88d5180749d1
Summary:
Update copyright statements to "2016-present". This makes our updated lint
rules happy and complies with the recommended license header statement.
Reviewed By: wez, bolinfest
Differential Revision: D4433594
fbshipit-source-id: e9ecb1c1fc66e4ec49c1f046c6a98d425b13bc27
Summary:
Previously we'd emit an AttributeError. This diff
fixes things up so that we show the help when no arguments are provided.
Reviewed By: simpkins
Differential Revision: D4434060
fbshipit-source-id: 838b1cab47118d8517e4dc0c5fde17fc69752b13
Summary:
This lock protects all operations that change the location of existing inodes
(rename(), unlink(), and rmdir()). This is needed to avoid race conditions
during rename operations.
I also included some FIXME comments about properly updating location
information for unloaded inodes. These should be taken care of in a subsequent
diff.
Reviewed By: wez
Differential Revision: D4361197
fbshipit-source-id: 5aaa1b8250f196b23207be8f3c0fd7f2603e6ae8
Summary:
Update the getSHA1() thrift handler to get the file SHA1 values in parallel.
The inode lookup itself still happens serially at the moment. We need to
provide a future-based version of EdenMount::getFileInode() in the future, and
change all existing callers of it to use the Future-based version.
Reviewed By: wez
Differential Revision: D4361091
fbshipit-source-id: 1abbc16df8c3edf52959a82f16a7f59e5d6d038b
Summary:
This updates the InodePtr and InodeBase code to actually implement Inode
unloading and destruction.
At the moment we keep Inode objects as long as possible, and only unload them
during shutdown, or when the last reference to an unlinked inode goes away.
However, it should be straightforward to add on-demand unloading in the future
for Inodes that have not been accessed in a while. The
TreeInode::unloadChildrenNow() function provides a template for what this would
look like (it would simply need to be changed to check an access time when
doing on-demand unloading).
Reviewed By: wez
Differential Revision: D4360765
fbshipit-source-id: a46b355f0ac0c25f873a156e62af5184317de735
Summary:
Update EdenMount so that it cannot be destroyed directly, and must be destroyed
through a special destroy() function. This function is not implemented yet,
but it will delay actual destruction of the EdenMount until all Inode objects
in the mount have been destroyed.
This diff primarily updates the users of EdenMount to call destroy() properly.
I will send a subsequent diff that implements destroy() at the same time as
implementing Inode unloading.
Reviewed By: wez
Differential Revision: D4360558
fbshipit-source-id: 202826b63b75e1de2b73270806da094206108a47
Summary:
This defines our own custom smart pointer type for Inode objects. This will
provide us with more control over Inode lifetime, allowing us to decide if we
want to unload them immediately when they become unreferenced, or keep them
around for a while.
This will also allow us to fix some memory management issues around EdenMount
destruction. Currently we destroy the EdenMount immediately when it is
unmounted. This can cause issues if other parts of the code are still holding
references to Inode objects from this EdenMount. Our custom InodePtr class
will also allow us to delay destroying the EdenMount until all of its Inodes
have been destroyed.
This diff adds the new pointer types and updates the code to use them, but does
not actually implement destroying unreferenced inodes yet. The logic for that
has proven to be slightly subtle; I will split it out into its own separate
diff.
Reviewed By: wez
Differential Revision: D4351072
fbshipit-source-id: 7a9d81cbd226c9662a79a2f2ceda82fe2651f312
Summary: Folly will soon be dropping support for GCC 4.8, which means that `folly::make_unique` will be going away, so codemod away as much as possible before then.
Reviewed By: yfeldblum
Differential Revision: D4411221
fbshipit-source-id: 31d61425f6595d0750ea2e4c95c7d42bb5a5d955
Summary: I fixed an unrelated bug in HTE that made some more paths lintable - and some more move-abuses visible :-(
Reviewed By: meyering
Differential Revision: D4397732
fbshipit-source-id: 560db05f64ecf160e8198caf80dbd988e0dad75d
Summary:
I ported the C++ test I wrote for `hg add` in D4317871 as an integration test.
Note that this caught a real bug in the Hg extension!
Reviewed By: wez
Differential Revision: D4385970
fbshipit-source-id: 97ee555d463f22af54791f334f9349d88861f45a
Summary:
This constructor is unsafe. The check it uses before doing the comparison isn't a safe enough check to see if the cast is valid. For example, this is broken in the presence of multiple inheritance because it doesn't adjust the pointer offset to the correct vtable.
e.g.
struct A {
virtual ~A() {};
virtual void doSomething() = 0;
};
struct B {
virtual ~B() {}
virtual void doSomethingElse() = 0;
};
struct C : public B, public A {
virtual ~C() {}
void doSomething() override {
std::cout << "Something!" << std::endl;
}
void doSomethingElse() override {
std::cout << "Something Else!" << std::endl;
}
};
int main (int argc, char **argv) {
auto c = folly::makeFuture<std::shared_ptr<C>>(std::make_shared<C>());
folly::Future<std::shared_ptr<A>> a = std::move(c);
a.get()->doSomething();
return 0;
}
This code will print "Something else!" when run.
Reviewed By: siyengar
Differential Revision: D3679673
fbshipit-source-id: dcbf40ca82d458f17ee11191591f8b8daf58c919
Summary:
If the EDEN_DAEMON_ARGS environment variable is set when running integration
tests, append it to the edenfs daemon command line.
For instance, this makes it easy to pass in a "--vmodule" flag to turn up
verbose logging in particular parts of the code.
Reviewed By: bolinfest
Differential Revision: D4359306
fbshipit-source-id: 6b549ac7220f6c8d7a1dfad3827921d6462d0f77
Summary:
Add VLOG() statements for every FUSE entry point. Turning up the VLOG level in
EdenDispatcher now allows us to have a record of all FUSE calls made using
inode numbers. (Calls involving file handles or directory handles currently go
directly to the handle in question and do not pass through the EdenDispatcher.
We should probably add log statements for those too, but that can wait for a
later diff.)
Reviewed By: bolinfest
Differential Revision: D4359066
fbshipit-source-id: b378d829c39fdb61faf63d2d400f3ff556c376e8
Summary:
This member variable was not updated properly when files were renamed.
InodeBase now tracks our parent properly, so we don't need our own copy.
This does still call getParentBuggy() (which does not perform proper locking)
in a couple places for performing overlay operations. We'll need to fix this
later when addressing other overlay concurrency handling issues.
Reviewed By: bolinfest
Differential Revision: D4348481
fbshipit-source-id: 19c1ffced6f63e1ff041d0bab2363fecdb93d5a3
Summary:
Have FileData objects store a pointer to the FileInode that owns them, rather
than just to the EdenMount.
FileData objects still store a direct pointer to their FileInode's mutex_ and
entry_. It's potentially worth just accessing these through inode_ in the
future.
Reviewed By: bolinfest
Differential Revision: D4348103
fbshipit-source-id: 1f8497979bfc89c6a192ca0195209335db0d911c
Summary:
Add an EdenMount* member variable to InodeBase. Previously each TreeInode kept
a pointer to its EdenMount, and this moves that into the InodeBase class.
This is needed to support upcoming diffs that will change InodeBase memory
management. InodeBase objects will be responsible for notifying the InodeMap
when they become unreferenced.
Reviewed By: bolinfest
Differential Revision: D4348079
fbshipit-source-id: 6bf7ea908c6096aa2bca5b21290c09cbd58d5af7
Summary:
This updates all of the eden code to use the new InodeMap class. This replaces
the InodeNameManager class and the unordered_map previously stored in the
EdenDispatcher.
Reviewed By: bolinfest
Differential Revision: D4325750
fbshipit-source-id: d80ae7581ba79ca2b63155e184995a3e83e85dc1
Summary:
This diff starts adding a new InodeMap class. This class will eventually
consolidate the functionality of InodeNameMap plus the inode map stored in
EdenDispatcher.
This new class will bring several new benefits:
- All inode mapping logic consolidated into a single class, with a single lock
protecting the maps.
- A well-defined model for loaded vs unloaded inodes. InodeMap explicitly
tracks inodes that have InodeBase objects created for them vs inodes that
have an inode number allocated (for FUSE) but do not have an InodeBase object
in memory. This will make it possible to unload Inode objects on demand to
reduce memory usage.
- Tracking of pending loads, and de-duplicating load requests. This ensures
that only one Inode object ever exists for a given inode number / path. If a
second request to load an Inode arrives when a previous load request is still
in progress, InodeMap deals with this situation properly.
- Better support for using Inode objects without FUSE. With the old code,
attempts to interact with Inode objects without going through the FUSE
dispatch (say, when processing a thrift call) could result in inconsistent
state. New inodes created would not be put into the EdenDispatcher map,
which could result in problems.
- More convenient child inode access from TreeInode. With this change, the
TreeInode class can easily tell which of its children are loaded. This makes
it easier to do tasks which only need to operate on existing loaded inode
state (for instance, dirstate computation).
- Support for saving and loading state, to implement graceful restart..
InodeMap provides a central place to write out inode state on shutdown and
restoring it on startup. Saved inodes can easily be restored to an
"unloaded" state on startup. This code is not implemented yet as part of
this diff, but it should be straightforward to add in future diffs.
Reviewed By: bolinfest
Differential Revision: D4318060
fbshipit-source-id: d9b16430fc8367e3516e788d1e991e5163aa6997
Summary:
This introduces an `HgExtensionTestBase` class that can be subclassed
to create tests for Mercurial integration with Eden. It also introduces
`status_test.py`, which includes some basic integration tests for `hg status`
in Eden. Although our existing `.cpp` tests give us a lot of coverage, they
do not verify the logic in the Python code for our Mercurial extension,
so this will help me sleep better at night.
Note that `HgExtensionTestBase` is a subclass of `testcase.EdenHgTest`
rather than using the `testcase.eden_repo_test` decorator because these
tests will only be used in the Mercurial case for now, so there is no point in
trying to run them when creating the mount from a Git repo, as well.
Further note that this also required a small change to `eden/fs/integration/lib/testcase.py`
because the Eden directory was not being created at `$HOME/local/.eden`, which
is something that the Mercurial extension relies on.
Reviewed By: simpkins
Differential Revision: D4337660
fbshipit-source-id: ea825134420ae6eb794d37b6c75da19a6aadaae5
Summary:
This preserves the existing behavior while making `REPO_ROOT` and `BUCK_OUT`
available to other test code.
Reviewed By: simpkins
Differential Revision: D4358875
fbshipit-source-id: a7b2cafe7e56673831beac59a85fcab436e637a5
Summary:
We can use `//` exclusively because we always build Eden with Buck and never
fbbuild, our legacy build system for fbcode.
This revision was initially created by running:
```
find eden -name TARGETS | xargs sed -i -e 's#@/#//#g'
```
And then manually updating the `DEFS` file now that we no longer need
some normalization code for an outdated pattern.
But then I got annoyed by other inconsistencies, so I went through and
alpha-sorted some lists, replaced all double quotes with single quotes,
and fixed indents to be two spaces.
Reviewed By: simpkins
Differential Revision: D4356724
fbshipit-source-id: ab07a48f12fa937c257213d12331efdf09e42da6
Summary:
Rename the existing TestBackingStore class to FakeBackingStore, and fill it out
with an implementation that allows test code to control the store.
The test code can populate the store with Trees and Blobs to return, and can
control when the Futures returned by the store are fulfilled.
Reviewed By: bolinfest
Differential Revision: D4338577
fbshipit-source-id: 79221b04d844bd6011078b799e55182de4ccdfdc
Summary:
Upstream mercurial commit d83ca854fa21bef17bf3f097087c691c7d07bb5b changed the
mercurial.ui API, breaking existing scripts using it.
ui objects now have to be created using the ui.load() classmethod, rather than
by constructing them directly. (Creating a ui object directly no longer loads
the global or user configs.)
Reviewed By: bolinfest
Differential Revision: D4354629
fbshipit-source-id: 650db3abab9f2eebaefa2b4d8029c169ccc30217
Summary:
This is a first diff to remove the "easiest" unused includes in fbcode.
* For safety, we only touch .cpp files without #if and #define,
* We do not try to remove redundant systems headers (aka. "packing").
The diff was generated as follows:
```
foundation/scripts/ls-cpp-dirs | grep -v '^\(\.\.\|external/\|.*/external\)' | xargs ffmr -o /tmp/ffmr-diff-1 codegraph/scripts/ffmr/analyze_includes_no_headers_no_packing_skipping_ifdefs.sh
cat /tmp/ffmr-diff-1/*.diff | patch -p2
hg commit -m something
arc diff --prepare --nolint --nounit --less-context --excuse refactoring
```
Note: `grep -v` is just an optimization. The actual configuration is in these two files:
diffusion/FBS/browse/master/fbcode/codegraph/analysis/config.py
diffusion/FBS/browse/master/fbcode/codegraph/scripts/ffmr/analyze_includes_no_headers_no_packing_skipping_ifdefs.sh
See the task for more context, and the recent "safety" improvements on the tool.
depends on D4317825 for very few cases where `nolint` had to be manually added.
Reviewed By: igorsugak
Differential Revision: D4312617
fbshipit-source-id: ecc1f0addfd0651fa4770fcc43cd1314661a311a
