Summary:
This allows detecting changes to the edenrc configuration
and adjusting how we read data accordingly.
To avoid a huge gnarly diff with a lot of dependencies, this diff
introduces a little interface class that defines the method to obtain
a possibly reloaded EdenConfig instance.
This diff shouldn't change any functionality.
Reviewed By: chadaustin
Differential Revision: D12813051
fbshipit-source-id: 35390dc28671ba46a03d098c4f280e2f567dbdc9
Summary:
Because Mercurial blob IDs change without the contents changing, and
because files get unloaded upon checkout, rebasing across a large
distance in history can result in status fetching a lot of
metadata. Keep a smallish LRU cache for SHA-1 and size by blob ID.
Reviewed By: strager
Differential Revision: D10419965
fbshipit-source-id: 81499573814775471913db05f924767c3bab300e
Summary:
There's a lot of logic here that need not be offloaded to the
importer threads
Reviewed By: chadaustin
Differential Revision: D9539811
fbshipit-source-id: 1379eef390df6400291a61263d003b493a474b81
Summary:
This test seems to fail frequently while waiting on the tree data for the
"src" tree (at line 182). The code previously was using a 1s timeout for this
operation; this changes it to use a 10s timeout.
I'm not sure what would have changed that makes this take longer now. It's
possible something changed with the treemanifest code on the mercurial side.
It is somewhat surprising to me that this would regularly take more than 1s
though since this is only computing data locally about a pretty small test
repository.
Reviewed By: wez
Differential Revision: D9583645
fbshipit-source-id: 24695cc1b940540ec32461e2d80c30a3fd87a3e3
Summary:
Add a new unit test that exercises HgImporter and HgBackingStore using an
alternate hg_import_helper.py script that can inject errors into the response
data stream. This tests the behavior of HgImporter when there are
serialization errors with the data, as opposed to legitimately-serialized
error responses containing a python exception message.
Reviewed By: wez
Differential Revision: D9510568
fbshipit-source-id: ae86f69d5559701e9e8a9a18569986d64488eaf4
Summary:
Change HgImporter::importFileContents() to return a Blob object rather than
just the IOBuf.
This shouldn't have any behavior changes. However, it might help track down
some of the cases where we have seen files get incorrectly imported with empty
contents in some rare cases. We're not sure what is causing that, but
incorrectly using a moved-from IOBuf object would be one thing that might
produce this behavior. By changing this code to a `unique_ptr<Blob>` will
will eliminate the possibility of seeing empty buffer contents if some code
path is somehow ever able to use one of these objects after they have been
moved away from. (The code would end up with a null pointer in this case
rather than an empty buffer. If this ever does occur in practice a null
pointer would make the problem more obvious than an empty buffer.)
Reviewed By: chadaustin
Differential Revision: D9494651
fbshipit-source-id: 799cc1b1d0e2ac32ae2013dc80c84ec17cc1c491
Summary:
To improve the determinism of our C++ tests, I am planning to switch
TestMount to a ManualExecutor. This adds a ManualExecutor constructor
to UnboundedQueueExecutor.
In Rust, I'd use a trait, but a simple class with two constructors works fine.
Reviewed By: strager
Differential Revision: D8846553
fbshipit-source-id: c52752105255503d26f1e65494c32b3f62882e44
Summary:
D8065370 changed the HgImporter code to make a blocking `get()` call on a
future from the hg importer thread pool. This caused deadlocks, since all of
the hg importer threads could become stuck waiting on these `get()` calls to
complete. These would be waiting on RocksDbLocalStore threads which were in
turn all busy waiting to schedule operations on the HgImporter threads.
This fixes the code to use `Future::then()` rather than `Future::get()` to
avoid blocking the HgImporter threads on these operations.
Reviewed By: wez
Differential Revision: D8438777
fbshipit-source-id: a0d647b10ef5a182be2d19f636c2dbc24eab1b23
Summary:
The `HgImporter::prefetchFiles()` method was previously sending the file
information to the `hg_import_helper.py` script as JSON. This caused
problems since the python JSON code decodes the path names as Unicode strings.
These then get passed to the mercurial code that expects binary data. It
tries encoding the unicode strings to ASCII, which can throw
UnicodeEncodeErrors.
This updates the code to use a simple binary encoding scheme, since we don't
really want to convert these pathnames from binary to unicode and back again.
The file names are not necessarily even guaranteed to be valid unicode data.
Reviewed By: chadaustin
Differential Revision: D8393757
fbshipit-source-id: 3ec4dcf2bea57e5400af94e9139d0636c446c1f0
Summary:
Update the HgRepo class to set the `HGRCPATH` environment variable to the
empty string to ensure that we always use consistent settings, and are not
affected by the current system's hgrc files.
Reviewed By: chadaustin
Differential Revision: D8410265
fbshipit-source-id: 477a3721860c067f39fbef4bb7d3d14cae3a14b6
Summary:
When testing D8108649 I accidentally deleted all of my trees
but didn't delete my commit2tree mapping. This diff allows Eden to
recover from that situation.
Reviewed By: wez
Differential Revision: D8108728
fbshipit-source-id: 94a9393294ca259303026c297683dac4b3ecfac4
Summary:
Preparation for adding an HgBackingStore test.
Per @[1077184107:simpkins], use_hg_tree_manifest defaults on and folly::logging is automatically initialized now, so we can just remove main() entirely.
Reviewed By: wez
Differential Revision: D8226799
fbshipit-source-id: 5b5038bc311f62c1ef9ed943f57ce8f7d68bdc89
Summary:
Remove getTreeFuture and have getTree always return a Future. It's a
little unfortunate that this will always allocate, but it sounds like
we might decide to put all RocksDB access on a background thread to
increase CPU parallelism.
Reviewed By: bolinfest
Differential Revision: D8101430
fbshipit-source-id: e12b7ab07b3468114a58753768655c107265b8af
Summary:
Remove getBlobFuture and have getBlob always return a Future. It's a
little unfortunate that this will always allocate, but it sounds like
we might decide to put all RocksDB access on a background thread to
increase CPU parallelism.
Reviewed By: bolinfest
Differential Revision: D8101402
fbshipit-source-id: d6cbbd7fe4fe55bad661c9158297db2f03f7d352
Summary:
Update the folly::Init code to define a `--logging` command line flag, and call
`folly::initLoggingOrDie()` with the value of this command line during
initialization.
This is similar to the existing code that initializes the glog library.
(Programs can use both glog and folly logging together in the same program, and
I expect that many programs will do so as parts get converted to folly::logging
and parts remain using glog.)
Reviewed By: yfeldblum
Differential Revision: D7827344
fbshipit-source-id: 8aa239fbad43bc0b551cbe40cad7b92fa97fcdde
Summary:
I got tired of typing PathComponentPiece{"..."} in tests so here are
some operator literals.
Reviewed By: simpkins
Differential Revision: D7956732
fbshipit-source-id: 85d9f3fd725853a54da9e70fc659bd7eb9e0862c
Summary:
Promote the folly logging code out of the experimental subdirectory.
We have been using this for several months in a few projects and are pretty
happy with it so far.
After moving it out of the experimental/ subdirectory I plan to update
folly::Init() to automatically support configuring it via a `--logging` command
line flag (similar to the initialization it already does today for glog).
Reviewed By: yfeldblum, chadaustin
Differential Revision: D7755455
fbshipit-source-id: 052db34c97f7516728f7cbb1a5ad959def2f6efb
Summary:
This removes the TARGETS files from the eden github repository. The
open source buck build has been failing for several months, since buck
removed support for the thrift_library() rule.
I will potentially take a stab at adding CMake build support for Eden
at some point in the future.
Reviewed By: chadaustin
Differential Revision: D6893233
fbshipit-source-id: e6023094a807cf481ac49998c6f21b213be6c288
Summary:
This is the type of a tree entry, which may be another tree, so
FileType is not an accurate name.
Reviewed By: simpkins
Differential Revision: D6981168
fbshipit-source-id: 997eb8a27f599310ed678ce221c8083722db8bff
Summary:
Our Model TreeEntry code was a bit too general - in reality, both git
and hg only support a handful of specific tree entries: regular files,
executable files, symlinks, and trees. (git also supports
submodules.) This diff delays the expansion of a TreeEntry's type
into a full mode_t.
Reviewed By: simpkins
Differential Revision: D6980003
fbshipit-source-id: 73729208000668078a180b728d7e0bb9169c6f3c
Summary:
This enables dropping in alternative implementations
of LocalStore and adds a MemoryLocalStore implementation for
use in our tests.
This diff doesn't change the default storage option for the
eden server. I'll look at adding such an option in a follow up diff.
Reviewed By: chadaustin
Differential Revision: D6910413
fbshipit-source-id: 018bf04e0bff101e1f0ab35e8580ca2a2622e5ef
Summary:
This is a codemod to change from using @/ to // in basic cases.
- TARGETS files with lines starting with @/ (but excluding @/third-party:
- autodeps lines in source and TARGETS files ( (dep|manual)=@/ ), excluding @/third-party
- Targets in string macros
The only thing left of the old format should be @/third-party:foo:bar
drop-conflicts
Reviewed By: ttsugriy
Differential Revision: D6605465
fbshipit-source-id: ae50de2e1edb3f97c0b839d4021f38d77b7ab64c
Summary:
Replace the initLoggingGlogStyle() function with a more generic initLogging()
function that accepts a log config string to be parsed with parseLogConfig().
Reviewed By: bolinfest, yfeldblum
Differential Revision: D6342086
fbshipit-source-id: fb1bffd11f190b70e03e2ccbf2b30be08d655242
Summary:
Update hg_import_helper.py to include the exception type name in error
responses. Add a new HgImportPyError class in the C++ code to include both the
python exception type name and the message string.
In the future this will give us the ability to perform special handling based
on the python exception type, rather than just on the message contents.
Reviewed By: bolinfest
Differential Revision: D6333613
fbshipit-source-id: 1074bbf9fa25ee8b1abeadc38b1a4f569bc18d13
Summary:
The gtest macros in this file were moved to folly/test/TestUtils.h
Update everything to just use folly/test/TestUtils.h directly.
Reviewed By: chadaustin
Differential Revision: D6301759
fbshipit-source-id: 7f2841c12d5bea15376f782fb3bf3bfef16039c7
Summary:
Per discussion with bolinfest, this brings Eden in line with clang-format.
This diff was generated with `find . \( -iname '*.cpp' -o -iname '*.h' \) -exec bash -c "yes | arc lint {}" \;`
Reviewed By: bolinfest
Differential Revision: D6232695
fbshipit-source-id: d54942bf1c69b5b0dcd4df629f1f2d5538c9e28c
Summary:
This refactors the treemanifest union store initialization code, and fixes a
crash introduced in D5560787 if --use_hg_tree_manifest is enabled but the
underlying repository does not support treemanifest.
This moves the treemanifest initialization code into a new importTreeManifest()
helper function, and separates it from parsing the CMD_STARTED response. We
now only initialize unionStore_ if FLAGS_use_hg_tree_manifest is enabled and
the repository supports treemanifest. Other places in the code now only check
if unionStore_ is non-null to see if they should try importing via
treemanifest.
Splitting importTreeManifest() from the CMD_STARTED response parsing will
potentially make things cleaner in the future for using multiple
hg_import_helper.py processes. Even if we start multiple of these processes,
we only need one instance of the C++ union store, and we don't need to
re-initialize the treemanifest separately each time.
Reviewed By: bolinfest
Differential Revision: D5647203
fbshipit-source-id: 71e14156f1d0ebc0880dd819c5b041b7c1146818
Summary:
This adds unit tests for the treemanifest import code, and fixes a couple of
bugs triggered by the tests: directory permissions were being set incorrectly,
and trees were being stored in the LocalStore with the wrong hash, causing
lookup errors later.
The HgImportTest code previously used HgImporter::importManifest(), which ended
up exercising only the flatmanifest code. This updates HgImporter to expose
separate importFlatManifest() and importTreeManifest() APIs, and updates the
tests to check both import mechanisms.
Reviewed By: wez
Differential Revision: D5365959
fbshipit-source-id: 3799ee3b937296e8025b666dd7176dbe7e268a31
Summary: Add unit tests for HgImporter to better test this import logic.
Reviewed By: bolinfest
Differential Revision: D5309171
fbshipit-source-id: b5619d8271fef1cc31a0f642ec2fcbf9eccced54
