sapling/eden/mononoke
Stanislau Hlebik b6bbd21600 mononoke: make it possible to quickly cancel derivation for a given repo
Summary:
We've had a few sevs where expensive derivation for a single repo overloaded
our backends. This diff adds a tunable that makes it possible to quickly cancel
all derivations for a given repo. And one important caveat - it doesn't only
cancel derivations for a given repo, it also cancels the initial commit
traversal stage which can be quite expensive.

Note that it cancells all derivations for a given repo, at the moment there's
no way to cancel deriving just a single derived data type. I'd like to add it,
but in the next diffs since it might require changing the tunables - they don't
support tuning a list of strings for a given repo. Besides, having a way to
quickly disable all derivations for a given repo might be useful anyway.

Reviewed By: farnz

Differential Revision: D29028444

fbshipit-source-id: 559917e0e25eb060fe5be77a412bfec4ac3f1779
2021-06-11 07:00:14 -07:00
..
admin rust: revert zstd crates 2021-06-11 04:39:54 -07:00
aliasverify rust: revert zstd crates 2021-06-11 04:39:54 -07:00
alpn rust: revert zstd crates 2021-06-11 04:39:54 -07:00
backfill_derived_data rust: revert zstd crates 2021-06-11 04:39:54 -07:00
benchmark_filestore rust: revert zstd crates 2021-06-11 04:39:54 -07:00
benchmarks rust: revert zstd crates 2021-06-11 04:39:54 -07:00
blobimport rust: revert zstd crates 2021-06-11 04:39:54 -07:00
blobimport_lib rust: revert zstd crates 2021-06-11 04:39:54 -07:00
blobrepo Make filenodes for merge commits more like unodes and less like Mercurial does 2021-06-11 04:53:09 -07:00
blobrepo_utils rust: revert zstd crates 2021-06-11 04:39:54 -07:00
blobstore rust: revert zstd crates 2021-06-11 04:39:54 -07:00
blobstore_healer rust: revert zstd crates 2021-06-11 04:39:54 -07:00
blobstore_sync_queue rust: revert zstd crates 2021-06-11 04:39:54 -07:00
bonsai_git_mapping rust: revert zstd crates 2021-06-11 04:39:54 -07:00
bonsai_globalrev_mapping rust: revert zstd crates 2021-06-11 04:39:54 -07:00
bonsai_hg_mapping rust: revert zstd crates 2021-06-11 04:39:54 -07:00
bonsai_svnrev_mapping rust: revert zstd crates 2021-06-11 04:39:54 -07:00
bookmarks rust: revert zstd crates 2021-06-11 04:39:54 -07:00
bulkops rust: revert zstd crates 2021-06-11 04:39:54 -07:00
cache_warmup rust: revert zstd crates 2021-06-11 04:39:54 -07:00
changesets rust: revert zstd crates 2021-06-11 04:39:54 -07:00
cmdlib rust: revert zstd crates 2021-06-11 04:39:54 -07:00
cmds rust: revert zstd crates 2021-06-11 04:39:54 -07:00
commit_rewriting rust: revert zstd crates 2021-06-11 04:39:54 -07:00
common rust: revert zstd crates 2021-06-11 04:39:54 -07:00
derived_data mononoke: make it possible to quickly cancel derivation for a given repo 2021-06-11 07:00:14 -07:00
edenapi_server rust: revert zstd crates 2021-06-11 04:39:54 -07:00
edenapi_service rust: revert zstd crates 2021-06-11 04:39:54 -07:00
fastreplay rust: revert zstd crates 2021-06-11 04:39:54 -07:00
filenodes rust: revert zstd crates 2021-06-11 04:39:54 -07:00
filestore rust: revert zstd crates 2021-06-11 04:39:54 -07:00
git rust: revert zstd crates 2021-06-11 04:39:54 -07:00
gotham_ext rust: revert zstd crates 2021-06-11 04:39:54 -07:00
hgcli rust: revert zstd crates 2021-06-11 04:39:54 -07:00
hgproto rust: revert zstd crates 2021-06-11 04:39:54 -07:00
hook_tailer rust: revert zstd crates 2021-06-11 04:39:54 -07:00
hooks rust: revert zstd crates 2021-06-11 04:39:54 -07:00
lfs_import rust: revert zstd crates 2021-06-11 04:39:54 -07:00
lfs_import_lib rust: revert zstd crates 2021-06-11 04:39:54 -07:00
lfs_protocol rust: revert zstd crates 2021-06-11 04:39:54 -07:00
lfs_server rust: revert zstd crates 2021-06-11 04:39:54 -07:00
load_limiter rust: revert zstd crates 2021-06-11 04:39:54 -07:00
manifest rust: revert zstd crates 2021-06-11 04:39:54 -07:00
megarepo_api rust: revert zstd crates 2021-06-11 04:39:54 -07:00
mercurial rust: revert zstd crates 2021-06-11 04:39:54 -07:00
metaconfig rust: revert zstd crates 2021-06-11 04:39:54 -07:00
microwave rust: revert zstd crates 2021-06-11 04:39:54 -07:00
mononoke_api rust: revert zstd crates 2021-06-11 04:39:54 -07:00
mononoke_api_hg rust: revert zstd crates 2021-06-11 04:39:54 -07:00
mononoke_hg_sync_job rust: revert zstd crates 2021-06-11 04:39:54 -07:00
mononoke_types rust: revert zstd crates 2021-06-11 04:39:54 -07:00
mutable_counters rust: revert zstd crates 2021-06-11 04:39:54 -07:00
newfilenodes rust: revert zstd crates 2021-06-11 04:39:54 -07:00
observability rust: revert zstd crates 2021-06-11 04:39:54 -07:00
permission_checker rust: revert zstd crates 2021-06-11 04:39:54 -07:00
phases rust: revert zstd crates 2021-06-11 04:39:54 -07:00
pushrebase rust: revert zstd crates 2021-06-11 04:39:54 -07:00
pushrebase_mutation_mapping rust: revert zstd crates 2021-06-11 04:39:54 -07:00
quiet_stream rust: revert zstd crates 2021-06-11 04:39:54 -07:00
reachabilityindex Make filenodes for merge commits more like unodes and less like Mercurial does 2021-06-11 04:53:09 -07:00
regenerate_hg_filenodes rust: revert zstd crates 2021-06-11 04:39:54 -07:00
repo_attributes rust: revert zstd crates 2021-06-11 04:39:54 -07:00
repo_client rust: revert zstd crates 2021-06-11 04:39:54 -07:00
repo_factory rust: revert zstd crates 2021-06-11 04:39:54 -07:00
repo_import rust: revert zstd crates 2021-06-11 04:39:54 -07:00
revset Make filenodes for merge commits more like unodes and less like Mercurial does 2021-06-11 04:53:09 -07:00
scs rust: revert zstd crates 2021-06-11 04:39:54 -07:00
scs_server Implement rust.name and emit raw identifiers for rust keywords 2021-06-10 10:52:36 -07:00
segmented_changelog Make filenodes for merge commits more like unodes and less like Mercurial does 2021-06-11 04:53:09 -07:00
server rust: revert zstd crates 2021-06-11 04:39:54 -07:00
sshrelay rust: revert zstd crates 2021-06-11 04:39:54 -07:00
statistics_collector rust: revert zstd crates 2021-06-11 04:39:54 -07:00
streaming_clone rust: revert zstd crates 2021-06-11 04:39:54 -07:00
tests mononoke: add support for by repo vec of strings tunables 2021-06-11 05:53:24 -07:00
time_window_counter rust: revert zstd crates 2021-06-11 04:39:54 -07:00
tunables mononoke: make it possible to quickly cancel derivation for a given repo 2021-06-11 07:00:14 -07:00
unbundle_replay rust: revert zstd crates 2021-06-11 04:39:54 -07:00
upload_globalrevs rust: revert zstd crates 2021-06-11 04:39:54 -07:00
walker rust: revert zstd crates 2021-06-11 04:39:54 -07:00
Cargo.toml rust: revert zstd crates 2021-06-11 04:39:54 -07:00
README.md mononoke: add README.md and the missing pieces for supporting cargo (#13) 2020-02-13 00:12:36 -08:00

Mononoke

Mononoke is a next-generation server for the Mercurial source control system, meant to scale up to accepting thousands of commits every hour across millions of files. It is primarily written in the Rust programming language.

Caveat Emptor

Mononoke is still in early stages of development. We are making it available now because we plan to start making references to it from our other open source projects.

The version that we provide on GitHub does not build yet.

This is because the code is exported verbatim from an internal repository at Facebook, and not all of the scaffolding from our internal repository can be easily extracted. The key areas where we need to shore things up are:

  • Full support for a standard cargo build.
  • Open source replacements for Facebook-internal services (blob store, logging etc).

The current goal is to get Mononoke working on Linux. Other Unix-like OSes may be supported in the future