A Scalable, User-Friendly Source Control System.
Go to file
Stanislau Hlebik 5dbdffdfe7 mononoke: fix sharded filenodes
Summary:
Previously to get copy/move source we had to join `paths` and `fixedcopyinfo`
table. That worked fine when we had just one shard. However now we have many
shards, and join no longer works. The reason is that move source path is in a
different shard compared to move destination path, and join returns no data.

Consider this situation. shardA contains all the data for pathA, shardB
contains all the data for pathB. That means that sharded `paths` table will
have pathA in shardA and pathB in shardB. Then if file pathA was copied form
pathB, then `fixedcopyinfo` table in shardA contains a path_hash of pathB.
However joining shardA's `fixedcopyinfo` with shardA's `paths` to convert
path_hash to path fails because pathB is in shardB.

The only possible fix is to split fetching path_hash from `fixedcopyinfo` and
converting path_hash to path.

I don't think we'll be able to keep the logic with join that we have at the
moment. It would require us to have all paths on all shards which is
unfeasible because it'll make writes much slower.

Reviewed By: aslpavel

Differential Revision: D13690141

fbshipit-source-id: 16b5cae6f23c162bb502b65c208f3ca9e443fb04
2019-01-17 02:33:35 -08:00
apiserver reformat several files 2019-01-16 11:47:37 -08:00
async-compression async-compression: re-add support for zstd decompression with warnings 2018-09-07 09:53:25 -07:00
asyncmemo rust/stats: don't require users to import lazy_static 2018-12-02 14:34:39 -08:00
blobrepo mononoke: change the way file content blobs are hashed 2019-01-17 02:33:35 -08:00
blobrepo_utils mononoke: pass CoreContext down to blobstore 2018-12-04 11:40:15 -08:00
blobstore sqlblob: add counters for internal data and chunk memcache hit 2019-01-15 09:38:26 -08:00
blobstore-sync-queue blobstore healer: minimalistic implementation 2019-01-15 10:41:07 -08:00
bonsai-hg-mapping make bulk select query code safer 2019-01-17 02:28:10 -08:00
bonsai-utils mononoke: pass CoreContext down to blobstore 2018-12-04 11:40:15 -08:00
bookmarks mononoke: back out shared-future-based bookmark cache 2019-01-14 07:09:23 -08:00
bundle2-resolver rustfmt (arc lint) for the file 2019-01-16 10:43:14 -08:00
bytes-ext add cargo build support for local development 2018-02-22 04:30:32 -08:00
cache-warmup/src mononoke: pass CoreContext down to blobstore 2018-12-04 11:40:15 -08:00
changesets mononoke: move RepositoryId into mononoke-types 2018-12-19 10:24:27 -08:00
cmdlib/src blobstore healer: minimalistic implementation 2019-01-15 10:41:07 -08:00
cmds blobstore healer: minimalistic implementation 2019-01-15 10:41:07 -08:00
common sql_ext: add PoolSizeConfig to enable easy configuring of MyRouter pool sizes 2019-01-15 09:38:26 -08:00
eden_server Add Cargo.toml files to crates. (#7) 2018-07-09 19:52:27 -07:00
failure_ext tp2/rust: update to rust-1.30.0 2018-11-06 16:17:18 -08:00
filenodes mononoke: fix sharded filenodes 2019-01-17 02:33:35 -08:00
futures-ext rust/futures-ext: implement StreamEither/left_stream/right_stream 2018-12-19 16:49:24 -08:00
hgcli mononoke: do not fail if same certificate was added twice 2018-09-27 10:37:00 -07:00
hgproto mononoke: avoid copies in encode_single 2019-01-16 09:21:44 -08:00
hook_tailer asynchronous blobrepo constructor 2019-01-14 09:31:58 -08:00
hooks Guard AclChecker::new and Identity::from_groupname as well from being included 2019-01-04 08:32:29 -08:00
mercurial mononoke: change how copy information is found 2019-01-17 02:33:35 -08:00
mercurial-bundles add time stats to phases calculation 2019-01-09 09:23:41 -08:00
mercurial-types mononoke: change how copy information is found 2019-01-17 02:33:35 -08:00
metaconfig blobstore healer: minimalistic implementation 2019-01-15 10:41:07 -08:00
mononoke-api/src mononoke: pass CoreContext down to blobstore 2018-12-04 11:40:15 -08:00
mononoke-types mononoke: change the way file content blobs are hashed 2019-01-17 02:33:35 -08:00
netstring/src rust/netstring: move encode and decode into separate modules. 2018-09-12 20:37:41 -07:00
phases calculate phases using bulk API 2019-01-16 08:22:54 -08:00
py_tar_utils move tar_utils.py from update_rust into an open sourceable location 2017-09-14 13:52:23 -07:00
reachabilityindex mononoke: make sure we don't have stackoverflows in reachability_query 2019-01-14 08:56:46 -08:00
ready_state/src server: split server binary crate into 4 separate crates 2018-07-17 04:54:58 -07:00
repo_client mononoke: change how copy information is found 2019-01-17 02:33:35 -08:00
revset Replacing HgNodeHash with Changesetid - Without formatting changes 2019-01-16 11:08:41 -08:00
server mononoke: Add PerfCounters struct to CoreContext 2019-01-16 09:11:17 -08:00
sshrelay rust/netstring: convert from error-chain to failure 2018-09-12 20:37:41 -07:00
tests mononoke: change the way file content blobs are hashed 2019-01-17 02:33:35 -08:00
.gitignore add .gitignore 2018-03-13 11:58:20 -07:00
.travis.yml Add travis-CI integration by adding .travis.yml file. 2018-07-26 10:09:32 -07:00
Cargo.toml Restore cachelib blob caching 2018-08-07 11:37:37 -07:00
CONTRIBUTING.md Initial commit 2017-07-27 18:00:19 -07:00
LICENSE Initial commit 2017-07-27 18:00:19 -07:00
packman.yml packman config file for admin tool 2018-06-04 22:09:49 -07:00
README.md mention in README that Mononoke supports Mercurial 2017-07-28 14:41:37 -07:00
rustfmt.toml Back out D13589633 "revert D13584553, D13175922, D13166085 to fix rust builds" 2019-01-09 13:31:19 -08:00

Mononoke

Mononoke is a next-generation server for the Mercurial source control system, meant to scale up to accepting thousands of commits every hour across millions of files. It is primarily written in the Rust programming language.

Caveat Emptor

Mononoke is still in early stages of development. We are making it available now because we plan to start making references to it from our other open source projects such as Eden.

The version that we provide on GitHub does not build yet.

This is because the code is exported verbatim from an internal repository at Facebook, and not all of the scaffolding from our internal repository can be easily extracted. The key areas where we need to shore things up are:

  • Full support for a standard cargo build.
  • Open source replacements for Facebook-internal services (blob store, logging etc).

The current goal is to get Mononoke working on Linux. Other Unix-like OSes may be supported in the future.