sapling/eden/mononoke
Simon Farnsworth a9b8793d2d Add a write-mostly blobstore mode for populating blobstores
Summary:
We're going to add an SQL blobstore to our existing multiplex, which won't have all the blobs initially.

In order to populate it safely, we want to have normal operations filling it with the latest data, and then backfill from Manifold; once we're confident all the data is in here, we can switch to normal mode, and never have an excessive number of reads of blobs that we know aren't in the new blobstore.

Reviewed By: krallin

Differential Revision: D22820501

fbshipit-source-id: 5f1c78ad94136b97ae3ac273a83792ab9ac591a9
2020-08-03 04:36:19 -07:00
..
benchmark mononoke: add cmdlib argument to control cachelib zstd compression 2020-07-31 01:12:02 -07:00
blobimport_lib bookmarks: extract BundleReplayData from BookmarkUpdateReason 2020-07-10 04:50:24 -07:00
blobrepo mononoke: use WarmBookmarkCache in repo_client 2020-07-31 03:09:24 -07:00
blobrepo_utils Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
blobstore Add a write-mostly blobstore mode for populating blobstores 2020-08-03 04:36:19 -07:00
blobstore_sync_queue Move blobstore healer tests to their own file 2020-07-11 05:41:36 -07:00
bonsai_git_mapping Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
bonsai_globalrev_mapping Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
bonsai_hg_mapping Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
bookmarks mononoke/repo_import: add hg sync checker 2020-08-03 04:01:27 -07:00
bulkops Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
cache_warmup Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
changesets Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
cmdlib mononoke: add cmdlib argument to control cachelib zstd compression 2020-07-31 01:12:02 -07:00
cmds Add a write-mostly blobstore mode for populating blobstores 2020-08-03 04:36:19 -07:00
commit_rewriting mononoke/mononoke_x_repo_sync_job: make mononoke_x_repo_sync_job and related public (#40) 2020-07-31 09:02:33 -07:00
common rust: mem::replace without using return value is just an assignment 2020-07-31 18:38:35 -07:00
derived_data don't error out when path doesn't exist 2020-07-30 03:25:01 -07:00
edenapi_server edenapi_server: fix comment in errors.rs 2020-07-24 00:56:30 -07:00
fastreplay mononoke: use WarmBookmarkCache in repo_client 2020-07-31 03:09:24 -07:00
filenodes Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
filestore mononoke: add context to alias deserialize error handling 2020-07-21 06:11:46 -07:00
git Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
gotham_ext gotham_ext: move client hostname lookup into gotham_ext 2020-07-30 10:27:35 -07:00
hgcli Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
hgproto mononoke/{hgproto,mercurial_bundles}: eliminate O(N^2) behavior in decoding 2020-07-08 08:07:13 -07:00
hook_tailer Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
hooks bookmarks: extract BundleReplayData from BookmarkUpdateReason 2020-07-10 04:50:24 -07:00
lfs_import_lib autocargo: regenerate 2020-05-19 16:08:40 -07:00
lfs_protocol mononoke: try to compress values if they above cachelib limit 2020-06-20 01:05:54 -07:00
lfs_server gotham_ext: move client hostname lookup into gotham_ext 2020-07-30 10:27:35 -07:00
load_limiter Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
manifest Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
mercurial bookmarks: extract BundleReplayData from BookmarkUpdateReason 2020-07-10 04:50:24 -07:00
metaconfig Add a write-mostly blobstore mode for populating blobstores 2020-08-03 04:36:19 -07:00
microwave Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
mononoke_api mononoke: add allow_bookmark_update_delay 2020-07-31 03:09:24 -07:00
mononoke_hg_sync_job mononoke/hg_sync_job: make mononoke_hg_sync_job public (#37) 2020-07-30 02:52:56 -07:00
mononoke_types mononoke: add context to alias deserialize error handling 2020-07-21 06:11:46 -07:00
mutable_counters admin: add a subcommand to manipulate mutable_counters 2020-04-16 03:19:44 -07:00
newfilenodes add Mononoke integration tests CI (#26) 2020-07-16 12:16:10 -07:00
permission_checker mononoke/x509 identity: add OSS parsing of x509 certificates (#32) 2020-07-24 09:05:52 -07:00
phases bookmarks: extract BundleReplayData from BookmarkUpdateReason 2020-07-10 04:50:24 -07:00
pushrebase bookmarks: remove repo_id from Bookmarks methods 2020-07-10 04:50:25 -07:00
reachabilityindex mononoke: RFC skiplist with gaps 2020-07-27 01:33:57 -07:00
regenerate_hg_filenodes mononoke/regenerate_hg_filenodes: make regenerate_hg_filenodes public (#39) 2020-07-30 06:50:54 -07:00
repo_client mononoke: use WarmBookmarkCache in repo_client 2020-07-31 03:09:24 -07:00
repo_import mononoke/repo_import: add hg sync checker 2020-08-03 04:01:27 -07:00
revset Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
scs_server/src add descendants_of argument to commit_path_history 2020-07-23 07:34:52 -07:00
segmented_changelog Regenerate all Cargo.tomls after upgrade to futures 0.3.5 2020-07-06 20:49:43 -07:00
server mononoke: add allow_bookmark_update_delay 2020-07-31 03:09:24 -07:00
sshrelay Clean out unused deps from our TARGETS files 2020-04-14 03:38:11 -07:00
tests mononoke/repo_import: add hg sync checker 2020-08-03 04:01:27 -07:00
time_window_counter mononoke/lfs_server: popularity: add ODS counters, timeouts and tests 2020-07-15 03:39:23 -07:00
tunables mononoke: use WarmBookmarkCache in repo_client 2020-07-31 03:09:24 -07:00
unbundle_replay mononoke: check that repo is locked during unbundle process 2020-07-21 09:41:38 -07:00
walker Add a write-mostly blobstore mode for populating blobstores 2020-08-03 04:36:19 -07:00
Cargo.toml mononoke/mononoke_x_repo_sync_job: make mononoke_x_repo_sync_job and related public (#40) 2020-07-31 09:02:33 -07:00
README.md mononoke: add README.md and the missing pieces for supporting cargo (#13) 2020-02-13 00:12:36 -08:00

Mononoke

Mononoke is a next-generation server for the Mercurial source control system, meant to scale up to accepting thousands of commits every hour across millions of files. It is primarily written in the Rust programming language.

Caveat Emptor

Mononoke is still in early stages of development. We are making it available now because we plan to start making references to it from our other open source projects.

The version that we provide on GitHub does not build yet.

This is because the code is exported verbatim from an internal repository at Facebook, and not all of the scaffolding from our internal repository can be easily extracted. The key areas where we need to shore things up are:

  • Full support for a standard cargo build.
  • Open source replacements for Facebook-internal services (blob store, logging etc).

The current goal is to get Mononoke working on Linux. Other Unix-like OSes may be supported in the future