Summary:
This diff removes all occurrences of `extern crate` from //fbcode/common/rust// except within files included into bindgen code which are forced to use 2015 edition.
`#[macro_use]` attributes are replaced by 2018-style individual `use` of macros.
Reviewed By: jsgf
Differential Revision: D16179130
fbshipit-source-id: 25771eaec8e38043bfa1f9be267dc185758cd2d7
Summary:
Now that Rust macros can be `use`d like normal symbols, `stats` can
simply import the `lazy_static!` macro without requiring its users to do it.
Reviewed By: Imxset21
Differential Revision: D13281897
fbshipit-source-id: a6780fbace07dd784308e642d4a384322a17c367
Summary:
Add optionally and necessarily inlinable strings, and another
concurrent hashmap impl.
The asyncmemo changes are because of futures 0.1.23, which fixed an [unsoundness bug](15dbc97f30).
Reviewed By: kulshrax
Differential Revision: D8840579
fbshipit-source-id: 77d3e724310b727aeacc0c86a0353ea01e8fc679
Summary:
This is a series of patches which adds Cargo.toml files to all the crates and tries to build them. There is individual patch for each crate which tells whether that crate build successfully right now using cargo or not, and if not, reason behind that.
Following are the reasons why the crates don't build:
* failure_ext and netstring crates which are internal
* error related to tokio_io, there might be an patched version of tokio_io internally
* actix-web depends on httparse which uses nightly features
All the build is done using rustc version `rustc 1.27.0-dev`.
Pull Request resolved: https://github.com/facebookexperimental/mononoke/pull/7
Differential Revision: D8778746
Pulled By: jsgf
fbshipit-source-id: 927a7a20b1d5c9643869b26c0eab09e90048443e
Summary:
It seems that a single mutex on a cache is causing too much contention.
In this diff the Mutex<CacheHash> is split into Vec<Mutex<CacheHash>> and a `std::collections::hash_map::DefaultHasher` is used to compute the shard of a key.
Reviewed By: StanislavGlebik
Differential Revision: D8627675
fbshipit-source-id: cbb46ff26815862f447d15bb92bca4eafc720eac
Summary:
Memcache is going to provide a faster implementation of `is_present`
that relies on the put cachemutex. We lose all the benefits if `is_present` up
here ends up always calling `get` from lower layers - teach it not to do that.
Reviewed By: jsgf
Differential Revision: D8318151
fbshipit-source-id: 130d4cb17c530c8677bdd563806989c393b4c6cf
Summary:
We had an asyncmemo bug when keys were, for example, Strings. If we insert
String::new("key"), and then try to remove String::new("key"), then these keys
may have different weights because Strings may have different capacity.
This is not the case for values though, because inserted and deleted value is
always the same object.
Let's not take key sizes into account at all. That means that the real asyncmemo memory usage will be
higher, however values are a great deal bigger than keys, so it sholdn't be an
issue.
Also we are planning to do it anyway, because we want to split
asyncmemo hashmap into in-flight futures and completed futures, and in-flight
futures won't take key sizes into account.
Reviewed By: jsgf
Differential Revision: D8040684
fbshipit-source-id: 332dbe5663182dd84ea291ca4b33b41995ac0166
Summary:
parking_lot's primitives are apparently lighter-weight than using
standard (pthreads) ones, and are more or less drop-in replacements.
Reviewed By: farnz
Differential Revision: D7661246
fbshipit-source-id: c7558a15971bf5b30c4dadb9437586832cdad3d4
Summary: As with changesets and blobs, let's cache filenodes data
Reviewed By: jsgf
Differential Revision: D7831105
fbshipit-source-id: 334cb474f5cc3ef8dba0945d11273b2b3875e8ad
Summary:
Use asyncmemo to cache Changesets.
Unfortunately currently we are using separate asyncmemo cache, so we have to
specify the size for the caches separately. Later we'll have a single cache for
everything, and the number of config knobs will go down.
Reviewed By: lukaspiatkowski
Differential Revision: D7685376
fbshipit-source-id: efe8a3a95fcc72fab4f4af93564e706cd1540c2f
Summary:
Polling state doesn't help much. It only helps prevent a create/create race,
however in practice it happens very rarely. Even if it happens, it won't be a
big deal, because the same future will be created twice.
At the same time Polling state makes a lot of things broken - see the new
tests. There were a lot of situations that can prevent future from ever
resolving.
Let's remove this state.
Reviewed By: jsgf
Differential Revision: D7635175
fbshipit-source-id: d1314ffdb91aca9c3fc11d734ca35b8965150b9f
Summary:
Add CachingBlobstore that uses Asyncmemo as a cache.
Normal Blobstore returns Option<Bytes>. CachingBlobstore doesn't cache `None` response, because this blob may appear later in the blobstore, and we want clients to be able to access it.
Reviewed By: farnz
Differential Revision: D6579385
fbshipit-source-id: 746a5c222dd00a35c3353e76c2792fc8e2323381
Summary:
That prevents us from implementing, for example, Weight for Bytes. There are no
impl specialization in Rust yet, and since Bytes do not implement HeapSizeOf,
we can't implement Weight for Bytes.
However we want to use Asyncmemo with Bytes. So in order to fix the issue let's
remove overgeneric Weight implementation and add implementation for the most
basic things.
Reviewed By: jsgf
Differential Revision: D7639162
fbshipit-source-id: b81d7e50599fdd8bdb8f9903d7aedff71734cdf2
Summary:
Not clear why this is happening, and I can't repro on my devserver, so
this debug code it is.
Reviewed By: swolchok
Differential Revision: D7480116
fbshipit-source-id: 0c9d54e22ed8ca51b27796792b0cc2045fe88f45
Summary:
Asyncmemo re-implementation using Shared future. The reasons for doing it are:
1 Now it's safe to drop MemoFuture
2 Now there will be no deadlocks where one future evict another future from the
cache, and then the second future evicts the first futures and so on.
Since Shared returns SharedError, I had to do a hack to work around that. More
details in the code comments.
Reviewed By: farnz
Differential Revision: D7099852
fbshipit-source-id: e08cb859ef06f28339ac9314edb8e53e92480874
Summary:
Previously if we had two different tasks (or the same tasks but with different
Notify, as in
[FuturesUnordered](http://alexcrichton.com/futures-rs/futures/stream/futures_unordered/struct.FuturesUnordered.html)),
interchangebly polling the future, only one latest future completes, and all
other futures receive no `notify()` call and may never finish.
This diff fixes it by storing a list of tasks that ever polled a future.
Note that if future was polled twice by the same task, the task will be
recorded twice. This is not great, but unfortunately one can't compare Tokio's
tasks.
Reviewed By: lukaspiatkowski
Differential Revision: D6611511
fbshipit-source-id: 2742cb85b13a684699a13a874e36e17b29fb4480
Summary:
There was a bug when if the entry with the same key is present in the hash and
the hash is full, then some entry will be evicted even though it's not
necessary.
It adds one hash map lookup, but it shouldn't be a big problem
Reviewed By: jsgf
Differential Revision: D6702001
fbshipit-source-id: 1c8cadc5e4bad6f6d95279b21edf00ed99f62c49
Summary:
In the next diff we'll `task::current()` inside MemoFuture. It fails if no
task exists. Let's use spawn to create a Task for each future.
Reviewed By: jsgf
Differential Revision: D6611510
fbshipit-source-id: 95a825dff9714579e9b16e74f355c54d8fa83a24
Summary: Previously inserting the same key twice will add size of this key twice. This diff fixes it.
Reviewed By: jsgf
Differential Revision: D6602501
fbshipit-source-id: eb8f785296945bca1df4b6cd33bd3050a48e2174
Summary: Add a unit test for failing to compute a value.
Reviewed By: Imxset21
Differential Revision: D5717910
fbshipit-source-id: cb5b1560a539f4b99d87dc2f8fc0afd37f4b5b97
Summary:
Pass a reference to the cache to the fill function. This allows the
function to be recursive based on memoized values.
This also required quite a bit of restructuring to make sure that locks
and ownership are handled properly during recursive calls. Specifically,
a new `Slot` state - `Polling` - is used to indicate when a thread/task
is currently calling `.poll()` on the future. This contains a list of
futures Tasks which are interested in the state of the slot which can be
notified when it changes state.
Also removed unused Entry API code.
Reviewed By: sid0
Differential Revision: D5652704
fbshipit-source-id: 29cd3fe37d4eb9316235872b7e2e228bf10a016f
