Summary:
This introduces caching of filenodes to Memcache as in the old filenodes
implementation. The code is mostly was ported over from the existing filenodes
implementation, and converted to async / await. However, one key difference is
that the lookups happen once we hold the semaphore to talk to the underlying
MySQL shard.
The reason for this is:
- Reads to Memcache are really fast. They're often under 1ms. If you're going
to miss in Memcache and have to go to SQL, it won't make you much slower.
- Reads to Memcache are kinda expensive CPU-wise. Data in Memcache is
compressed, and we often see a lot of our CPU cycles spent talking to Memache
when we're under load.
- Memcache isn't an infinite resource. If we're reading the exact same
key a hundred times, that's going to hit the same Memcache box. A bit of
deduplication on our end is a nice thing to strive for. Besides, our own
thread pool we use to talk to Memcache is limited in size.
From a performance perspective, this doesn't make things any slower, but
reduces CPU usage when we'd otherwise have a lot of duplicate fetching.
Finally, note that this update also includes support for dirty-tracking in our
local cache. We use this to know if we should fill the remote cache (if we 100%
hit in local cache, we don't fill the remote cache).
Reviewed By: StanislavGlebik
Differential Revision: D19905390
fbshipit-source-id: 363f638bb24cf488c7cd3a8ecea43e93f8391d3f
Summary:
These comments end up being a source of churn as we roll out D20125635, and anyway are not particularly meaningful after the transformations performed by autocargo. For example:
```
bytes = { version = "0.4", features = ["serde"] } # todo: remove
```
^ This doesn't mean the generated Cargo.toml intends to drop its bytes dependency altogether, but just that will be migrated to a different version that is present in the third-party/rust/Cargo.toml but not visible in the generated Cargo.toml.
Reviewed By: jsgf
Differential Revision: D20128612
fbshipit-source-id: a9e7b29ddc4b26bc47a626dd73bdaa4771ee7b18
Summary:
This is the second (and last) step on removing RocksDB as a blobstore.
Check the task for more description.
Context for OSS:
> The issue with rocksblob (and to some extent sqlite) is that unless we
> introduce a blobstore tier/thift api (which is something I'm hoping to avoid
> for xdb blobstore) we'd have to combine all the mononoke function like hg,
> scs, LFS etc into one binary for it to have access to rocksdb, which would be
> quite a big difference to how we deploy internally
(Note: this ignores all push blocking failures!)
Reviewed By: farnz
Differential Revision: D20001261
fbshipit-source-id: c4b2b2a393b918d17680ad483aa1d77356f1d07c
Summary:
MyRouter needs to be told which shards to watch. Since I'm adding a new shard,
it'll be easier for everyone to know that they need to update their MyRouter
configuration if we start logging the shard name we're trying to hit.
Reviewed By: ikostia
Differential Revision: D20001704
fbshipit-source-id: 8a9ff3521bc7e3c9b7ed39c6ae33d0ddc1d467b7
Summary: common/sql_ext will now be buildable in OSS
Reviewed By: krallin
Differential Revision: D19876764
fbshipit-source-id: 0f51abd1169f6b8108e7e4cab85b5f193c28e2cd
Summary:
This commit manually synchronizes the internal move of
fbcode/scm/mononoke under fbcode/eden/mononoke which couldn't be
performed by ShipIt automatically.
Reviewed By: StanislavGlebik
Differential Revision: D19722832
fbshipit-source-id: 52fbc8bc42a8940b39872dfb8b00ce9c0f6b0800
Summary: create a bounded traversal crate with the 0.1 version of bounded_traversal_stream functionality, prior to updating to futures 0.3
Reviewed By: farnz
Differential Revision: D19804873
fbshipit-source-id: e98e00111fee5b1a9fcfc20bb054eeae1263fb26
