sapling/eden/mononoke
Thomas Orozco f4f96c1100 mononoke/microwave: create repository snapshots for faster cache warmup
Summary:
This introduces a new binary and library that (microwave: it makes warmup
faster..!) that can be used to accelerate cache warmup. The idea is the
microwave binary will run cache warmup and capture things that are loaded
during cache warmup, and commit those to a file.

We can then use that file when starting up a host to get a head start on cache
warmup by injecting all those entries into our local cache before actually
starting cache warmup.

Currently, this only supports filenodes, but that's already a pretty good
improvement. Changesets should be easy to add as well. Blobs might require a
bit more work.

Reviewed By: StanislavGlebik

Differential Revision: D20219905

fbshipit-source-id: 82bb13ca487f82ca53b4a68a90ac5893895a96e9
2020-03-04 04:02:18 -08:00
..
apiserver rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
benchmark mononoke/microwave: create repository snapshots for faster cache warmup 2020-03-04 04:02:18 -08:00
blobimport_lib/src rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
blobrepo rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
blobrepo_utils mononoke (+ rust/shed/async_unit): update async_unit to expect async fn's 2020-02-18 01:55:00 -08:00
blobstore rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
blobstore_sync_queue mononoke/blobstore_sync_queue: delete in smaller batches 2020-02-20 12:26:50 -08:00
bonsai_git_mapping rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
bonsai_globalrev_mapping rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
bonsai_hg_mapping rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
bookmarks rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
cache_warmup/src mononoke: add context with repoid to cache_warmup error message 2020-03-04 01:52:11 -08:00
changesets mononoke: convert places that talk to Memcache to Bytes 0.5 2020-02-26 03:30:47 -08:00
cmdlib rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
cmds rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
commit_rewriting rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
common rust: Move futures-old rdeps to renamed futures-old 2020-03-02 21:02:50 -08:00
derived_data rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
edenapi_server/src rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
fastreplay/src mononoke/fastreplay: add context on cache warmup failures 2020-03-04 03:14:45 -08:00
filenodes mononoke/microwave: create repository snapshots for faster cache warmup 2020-03-04 04:02:18 -08:00
filestore/src mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
git rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
gotham_ext/src rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
hgcli rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
hgproto rust: Begin upgrading to bytes 0.5 2020-02-18 21:20:09 -08:00
hook_tailer rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
hooks rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
lfs_import_lib/src mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
lfs_protocol autocargo: Strip line comments 2020-02-26 16:31:52 -08:00
lfs_server/src rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
manifest rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
mercurial mercurial_types: make envelope functions use generics instead of trait objects 2020-03-03 15:11:04 -08:00
metaconfig RFC: mononoke: introduce unodes v2 2020-03-02 05:27:31 -08:00
microwave mononoke/microwave: create repository snapshots for faster cache warmup 2020-03-04 04:02:18 -08:00
mononoke_api/src mercurial_types: make envelope functions use generics instead of trait objects 2020-03-03 15:11:04 -08:00
mononoke_types RFC: mononoke: introduce unodes v2 2020-03-02 05:27:31 -08:00
newfilenodes mononoke/microwave: create repository snapshots for faster cache warmup 2020-03-04 04:02:18 -08:00
phases rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
pushrebase/src rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
reachabilityindex rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
repo_client rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
revset rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
scs_server/src rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
segmented_changelog/src rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
server rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
sshrelay autocargo: Strip line comments 2020-02-26 16:31:52 -08:00
tests rust: Rename futures_preview:: to futures:: 2020-03-03 11:01:20 -08:00
walker/src mononoke/walker: disable filenodes SQL timeouts 2020-03-04 03:20:26 -08:00
Cargo.toml autocargo: Strip line comments 2020-02-26 16:31:52 -08:00
README.md mononoke: add README.md and the missing pieces for supporting cargo (#13) 2020-02-13 00:12:36 -08:00

Mononoke

Mononoke is a next-generation server for the Mercurial source control system, meant to scale up to accepting thousands of commits every hour across millions of files. It is primarily written in the Rust programming language.

Caveat Emptor

Mononoke is still in early stages of development. We are making it available now because we plan to start making references to it from our other open source projects.

The version that we provide on GitHub does not build yet.

This is because the code is exported verbatim from an internal repository at Facebook, and not all of the scaffolding from our internal repository can be easily extracted. The key areas where we need to shore things up are:

  • Full support for a standard cargo build.
  • Open source replacements for Facebook-internal services (blob store, logging etc).

The current goal is to get Mononoke working on Linux. Other Unix-like OSes may be supported in the future