sapling/eden/mononoke
Thomas Orozco 149e15f2ad mononoke: use spawn_future in getpack to fetch history
Summary:
The former implementation would eagerly query Memcache when fetching history
(due to how old futures work) for files in getpack, but the new one does not.
This means the new one loses out on a lot of buffering, which the old one used
to do.

This diff emulates the old behavior by eagerly querying filenodes in getpack,
which improves performance on a very big getpack (32K files) by about 3x, and
makes it 30% faster than the old code, instead of > 2x slower.

Note that I'm not certain we really want to do this kind of aggressive
buffering in getpack long term, but for now, I'd like to keep this unchanged.

Reviewed By: StanislavGlebik

Differential Revision: D19905398

fbshipit-source-id: 49f9a2cd505a98123fd1dabb835e8e378d45c930
2020-02-27 12:34:21 -08:00
..
apiserver mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
benchmark mononoke/filenodes: expect a Vec of filenodes to insert 2020-02-27 12:34:18 -08:00
blobimport_lib/src make BonsaiGitMapping repo-specific 2020-02-27 08:05:13 -08:00
blobrepo mononoke: switch to new filenodes implementation 2020-02-27 12:34:20 -08:00
blobrepo_utils mononoke (+ rust/shed/async_unit): update async_unit to expect async fn's 2020-02-18 01:55:00 -08:00
blobstore mononoke: switch to new filenodes implementation 2020-02-27 12:34:20 -08:00
blobstore_sync_queue mononoke/blobstore_sync_queue: delete in smaller batches 2020-02-20 12:26:50 -08:00
bonsai_git_mapping make BonsaiGitMapping repo-specific 2020-02-27 08:05:13 -08:00
bonsai_globalrev_mapping mononoke: add lease renewing to derived data 2020-02-25 09:22:46 -08:00
bonsai_hg_mapping mononoke: convert places that talk to Memcache to Bytes 0.5 2020-02-26 03:30:47 -08:00
bookmarks mononoke/warm_bookmarks_cache: use the repo's configuration 2020-02-18 07:40:25 -08:00
cache_warmup/src derived_data: add DeriveError for when derivation is disabled 2020-02-19 09:28:09 -08:00
changesets mononoke: convert places that talk to Memcache to Bytes 0.5 2020-02-26 03:30:47 -08:00
cmdlib mononoke: switch to new filenodes implementation 2020-02-27 12:34:20 -08:00
cmds mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
commit_rewriting mononoke: add lease renewing to derived data 2020-02-25 09:22:46 -08:00
common mononoke/newfilenodes: introduce remote caching 2020-02-27 12:34:19 -08:00
derived_data mononoke/filenodes: expect a Vec of filenodes to insert 2020-02-27 12:34:18 -08:00
edenapi_server/src edenapi_server: move handlers to submodule 2020-02-19 09:59:14 -08:00
fastreplay/src mononoke: include DB config in multiplexed blobstore configuration 2020-02-24 11:54:45 -08:00
filenodes mononoke/newfilenodes: introduce remote caching 2020-02-27 12:34:19 -08:00
filestore/src mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
git mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
gotham_ext/src gotham_ext: move TLS-related middleware into gotham_ext 2020-02-13 17:47:34 -08:00
hgcli Manual synchronization of fbcode/eden and facebookexperimental/eden 2020-02-11 11:42:43 +01:00
hgproto rust: Begin upgrading to bytes 0.5 2020-02-18 21:20:09 -08:00
hook_tailer Manual synchronization of fbcode/eden and facebookexperimental/eden 2020-02-11 11:42:43 +01:00
hooks mononoke: add a @nocommit hook 2020-02-27 08:28:05 -08:00
lfs_import_lib/src mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
lfs_protocol autocargo: Strip line comments 2020-02-26 16:31:52 -08:00
lfs_server/src mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
manifest Manual synchronization of fbcode/eden and facebookexperimental/eden 2020-02-11 11:42:43 +01:00
mercurial mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
metaconfig mononoke: pull config schema changes from configerator 2020-02-27 05:01:50 -08:00
mononoke_api/src mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
mononoke_types mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
newfilenodes mononoke/newfilenodes: introduce timeouts talking to Memcache, MySQL 2020-02-27 12:34:20 -08:00
phases mononoke (+ rust/shed/async_unit): update async_unit to expect async fn's 2020-02-18 01:55:00 -08:00
pushrebase/src mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
reachabilityindex mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
repo_client mononoke: use spawn_future in getpack to fetch history 2020-02-27 12:34:21 -08:00
revset mononoke (+ rust/shed/async_unit): update async_unit to expect async fn's 2020-02-18 01:55:00 -08:00
scs_server/src mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
segmented_changelog/src mononoke (+ rust/shed/async_unit): update async_unit to expect async fn's 2020-02-18 01:55:00 -08:00
server autocargo: Strip line comments 2020-02-26 16:31:52 -08:00
sshrelay autocargo: Strip line comments 2020-02-26 16:31:52 -08:00
tests mononoke: update internals to Bytes 0.5 2020-02-27 08:08:28 -08:00
walker/src mononoke: walker: log per-run session id to scuba for scrub 2020-02-27 09:00:44 -08:00
Cargo.toml autocargo: Strip line comments 2020-02-26 16:31:52 -08:00
README.md mononoke: add README.md and the missing pieces for supporting cargo (#13) 2020-02-13 00:12:36 -08:00

Mononoke

Mononoke is a next-generation server for the Mercurial source control system, meant to scale up to accepting thousands of commits every hour across millions of files. It is primarily written in the Rust programming language.

Caveat Emptor

Mononoke is still in early stages of development. We are making it available now because we plan to start making references to it from our other open source projects.

The version that we provide on GitHub does not build yet.

This is because the code is exported verbatim from an internal repository at Facebook, and not all of the scaffolding from our internal repository can be easily extracted. The key areas where we need to shore things up are:

  • Full support for a standard cargo build.
  • Open source replacements for Facebook-internal services (blob store, logging etc).

The current goal is to get Mononoke working on Linux. Other Unix-like OSes may be supported in the future