Summary:
With Facebook having been renamed Meta Platforms, we need to change the license
headers.
Reviewed By: fanzeyi
Differential Revision: D33407812
fbshipit-source-id: b11bfbbf13a48873f0cea75f212cc7b07a68fb2e
Summary:
This contains several fixes to LocalStore handling during shutdown.
- Have EdenServer explicitly call localStore_->close() during shutdown.
This ensures that the local store really gets close, just in case some other
part of the code somehow still has an outstanding shared_ptr reference to
it.
- Add synchronization around internal access to the RocksDB object in
RocksDbLocalStore. This ensures that calling `close()` is safe even if
there happens to still be some outstanding I/O operations. In particular
this helps ensure that if background GC operation is in progress that
`close()` will wait until it completes before destroying the DB object.
This also improves the code so that calling subsequent methods on a closed
RocksDbLocalStore throws an exception, instead of simply crashing.
I don't believe the additional synchronization in RocksDbLocalStore should
have much impact on performance: the synchronization overhead should be very
low compared to the cost of the RocksDB reads/writes.
Ideally some of this synchronization logic should perhaps be moved into the
base `LocalStore` class: all of the different `LocalStore` implementations
should ideally ensure that `close()` is thread-safe and blocks until other
pending I/O operations are complete. However, that requires a bigger
refactoring. I may attempt that in a subsequent diff, but for now I mainly
want to address this problem just for RocksDbLocalStore.
Reviewed By: strager
Differential Revision: D15948382
fbshipit-source-id: 96d633ac0879b3321f596224907fcfe72691b3f0
Summary:
Update the copyright & license headers in CMake files to reflect the
relicensing to GPLv2
Reviewed By: wez
Differential Revision: D15487079
fbshipit-source-id: 715e559464c19a0070d6e55a095b3fc7d61ad2f8
Summary:
Update the copyright & license headers in C++ files to reflect the
relicensing to GPLv2
Reviewed By: wez
Differential Revision: D15487078
fbshipit-source-id: 19f24c933a64ecad0d3a692d0f8d2a38b4194b1d
Summary:
Add a mechanism to open the RocksDbLocalStore in read-only mode. This is
primarily helpful to allow the `eden_store_util` process to examine a DB
without making many modifications, including automatic compactions that might
happen otherwise.
Note that this doesn't appear to be entirely read-only from a file contents
perspective: RocksDB does still rewrite the MANIFEST and OPTIONS files when
opening the DB in read-only mode, and does rotate the logs and write a new LOG
file.
Reviewed By: wez
Differential Revision: D15350216
fbshipit-source-id: 797498f9575a05334b741322654843d6545ca35c
Summary:
This moves some logic from the RocksHandles class up to RocksDbLocalStore.
The main thing moved here is the logic to automatically try and repair the DB
if opening fails. This will make it easier in a subsequent diff to make the
repair logic a bit smarter and more aware of our column family semantics.
This keeps RocksHandles a pretty dumb wrapper around the RocksDB object and
column family handles, whose only purpose is to manage destroying these two
things in the correct order.
Reviewed By: chadaustin
Differential Revision: D15043208
fbshipit-source-id: ee2d5619ac7781a892e1ba151712eee9e3ebfb14
Summary:
we need to be explicit about pulling in the dep for the
case where folly is not installed into a default installation
prefix.
Reviewed By: strager, pkaush
Differential Revision: D14683955
fbshipit-source-id: 0f302877674fb744ef8076641cd3fa72de74efe4
Summary:
Update `RocksHandles` to call `RepairDB()` if we get an error opening the
database.
We have seen errors opening the DB in some cases after hard server reboots.
In every case so far `ldb repair` has been able to repair it with no adverse
effects. This simply makes `edenfs` automatically attempt to perform this DB
repair step.
Reviewed By: chadaustin, strager
Differential Revision: D14452216
fbshipit-source-id: 10c0cb0ff9cea3c3bbe485a4e489e4a6df640803
Summary:
This removes the TARGETS files from the eden github repository. The
open source buck build has been failing for several months, since buck
removed support for the thrift_library() rule.
I will potentially take a stab at adding CMake build support for Eden
at some point in the future.
Reviewed By: chadaustin
Differential Revision: D6893233
fbshipit-source-id: e6023094a807cf481ac49998c6f21b213be6c288
Summary:
This is a codemod to change from using @/ to // in basic cases.
- TARGETS files with lines starting with @/ (but excluding @/third-party:
- autodeps lines in source and TARGETS files ( (dep|manual)=@/ ), excluding @/third-party
- Targets in string macros
The only thing left of the old format should be @/third-party:foo:bar
drop-conflicts
Reviewed By: ttsugriy
Differential Revision: D6605465
fbshipit-source-id: ae50de2e1edb3f97c0b839d4021f38d77b7ab64c
Summary:
Per discussion with bolinfest, this brings Eden in line with clang-format.
This diff was generated with `find . \( -iname '*.cpp' -o -iname '*.h' \) -exec bash -c "yes | arc lint {}" \;`
Reviewed By: bolinfest
Differential Revision: D6232695
fbshipit-source-id: d54942bf1c69b5b0dcd4df629f1f2d5538c9e28c
Summary:
We're running down a performance problem with hg status that we believe
is something happening at a higher level in the code but noticed that there
were a lot of reads of the rocksdb SST files. In the strace output for those
we observed file content data being read. The status operation shouldn't
need file contents; what's happening is that we're over-fetching some metadata
but happen to be scooping up the file contents from the SST file because we
use the same key prefixes and differentiate the keyspace with key suffixes.
This diff implements the use of rocksdb column families to partition things
more effectively and results in a speed up of around 6x in this scenario.
Furthermore, applying point lookup optimization options yields an additional
2x performance improvement to our rocksdb performance.
As part of this diff, I've removed the hash set that we were using to allow
checking whether a key was present in the store; it wasn't very useful
and would have had to be split into one set per keyspace with this diff;
easier to just remove it.
Reviewed By: bolinfest
Differential Revision: D5781906
fbshipit-source-id: 97f068ade546fd09f391e60a7a57fec0e9081e67
Summary: It doesn't need to exist anymore
Reviewed By: yfeldblum
Differential Revision: D5318746
fbshipit-source-id: c70b184f4b3fc12ede4632d6b3d43de16ed758c7
Summary:
Format all of the TARGETS files under eden/fs with the autodeps tool.
A few rocksdb include statements require comments so that autodeps can
correctly tell which dependency this include comes from. The rocksdb library's
source file structure unfortunately does not match the layout of how its header
files get installed, so autodeps cannot figure this out automatically.
Reviewed By: wez
Differential Revision: D5316000
fbshipit-source-id: f8163adca79ee4a673440232d6467fb83e56aa10
Summary:
Update copyright statements to "2016-present". This makes our updated lint
rules happy and complies with the recommended license header statement.
Reviewed By: wez, bolinfest
Differential Revision: D4433594
fbshipit-source-id: e9ecb1c1fc66e4ec49c1f046c6a98d425b13bc27
Summary:
We can use `//` exclusively because we always build Eden with Buck and never
fbbuild, our legacy build system for fbcode.
This revision was initially created by running:
```
find eden -name TARGETS | xargs sed -i -e 's#@/#//#g'
```
And then manually updating the `DEFS` file now that we no longer need
some normalization code for an outdated pattern.
But then I got annoyed by other inconsistencies, so I went through and
alpha-sorted some lists, replaced all double quotes with single quotes,
and fixed indents to be two spaces.
Reviewed By: simpkins
Differential Revision: D4356724
fbshipit-source-id: ab07a48f12fa937c257213d12331efdf09e42da6
