A Scalable, User-Friendly Source Control System.
Go to file
Anastasiya Zhyrkevich 3919c0eb4c LFS verification tool
Summary:
Purpose:
- Sha256 alias link to file_content is a required part of LFS getfiles works correct.
LFS protocol uses SHA-256 to refer to the file content. Mononoke uses Blake2.
To support LFS in Mononoke we need to set up a link from SHA-256 hash of the content to blake2 of the content.
These links are called aliases.

- Aliases are uploading together with file content blobs.
But only for new push operations.
- If repo is blobimported from somewhere, we need to make sure, that all the links are in blobstore.
If repo was blobimported before aliases were added then it may miss aliases for some blobs.
- This tool can be used to
   - find if any aliases are missing
   - fill missing aliases.

Implementation:
- Run in repo.
- Iterate through all changesets.
- Go through all the file_content blobs in the changesets
- Verify/generate alias256 links to file_content blobs.

Mode supported:
- verify, count the number of errors and print to console
- generate, if blob is missing to add it to the blobstore

Reviewed By: StanislavGlebik

Differential Revision: D10461827

fbshipit-source-id: c2673c139e2f2991081c4024db7b85953d2c5e35
2018-11-08 03:41:40 -08:00
apiserver Remove /status endpoint 2018-11-07 10:56:07 -08:00
async-compression async-compression: re-add support for zstd decompression with warnings 2018-09-07 09:53:25 -07:00
asyncmemo Add travis-CI integration by adding .travis.yml file. 2018-07-26 10:09:32 -07:00
blobrepo LFS verification tool 2018-11-08 03:41:40 -08:00
blobrepo_utils Make get_manifest_by_nodeid accept HgManifestId 2018-10-11 06:50:16 -07:00
blobstore Make all bindings fully exception-safe 2018-08-07 13:37:08 -07:00
bonsai-hg-mapping mononoke: add mem write version of bonsai-hg-mapping 2018-11-06 04:11:51 -08:00
bonsai-utils bonsai-utils/diff: add ChangedReusedId state 2018-07-31 10:36:31 -07:00
bookmarks bookmarks: convert from diesel to sql 2018-10-29 05:18:29 -07:00
bundle2-resolver sha1 check for lfs 2018-10-25 05:56:21 -07:00
bytes-ext add cargo build support for local development 2018-02-22 04:30:32 -08:00
cache-warmup/src mononoke: cache warmup improvements 2018-09-28 09:37:00 -07:00
changesets LFS verification tool 2018-11-08 03:41:40 -08:00
cmdlib/src LFS verification tool 2018-11-08 03:41:40 -08:00
cmds LFS verification tool 2018-11-08 03:41:40 -08:00
common mononoke: remove diesel from codebase 2018-10-29 05:18:29 -07:00
docs CODEMOD: rename mercurial::Parents to HgParents 2018-04-16 03:40:25 -07:00
eden_server Add Cargo.toml files to crates. (#7) 2018-07-09 19:52:27 -07:00
failure_ext tp2/rust: update to rust-1.30.0 2018-11-06 16:17:18 -08:00
filenodes Make it possible to configure a sharded database 2018-11-05 10:28:40 -08:00
futures-ext rust-crates-io: add crossbeam to tp2 2018-10-08 21:36:00 -07:00
hgcli mononoke: do not fail if same certificate was added twice 2018-09-27 10:37:00 -07:00
hgproto tp2: update rust-crates-io 2018-10-15 23:08:01 -07:00
hook_tailer mononoke: allow to set limit on the number of processed commits 2018-10-22 05:22:20 -07:00
hooks tp2/rust: update to rust-1.30.0 2018-11-06 16:17:18 -08:00
hooks_old/src mononoke: use ChangesetId in Changesets 2018-08-06 10:36:43 -07:00
mercurial Get parents without reconstructing file 2018-10-31 11:37:59 -07:00
mercurial-bundles tp2/rust: update to rust-1.30.0 2018-11-06 16:17:18 -08:00
mercurial-types mononoke: remove diesel from codebase 2018-10-29 05:18:29 -07:00
metaconfig Make it possible to configure a sharded database 2018-11-05 10:28:40 -08:00
mononoke-api/src get/generate hg changeset from bonsai changeset 2018-07-31 11:36:14 -07:00
mononoke-types mononoke: remove diesel from codebase 2018-10-29 05:18:29 -07:00
netstring/src rust/netstring: move encode and decode into separate modules. 2018-09-12 20:37:41 -07:00
py_tar_utils move tar_utils.py from update_rust into an open sourceable location 2017-09-14 13:52:23 -07:00
reachabilityindex/src hadle change only in executable bit same way as Hg 2018-10-15 02:16:50 -07:00
ready_state/src server: split server binary crate into 4 separate crates 2018-07-17 04:54:58 -07:00
repo_client Advertise tree-only nature in our caps string 2018-11-06 04:03:15 -08:00
revset add stats collection in ChangesetFetcher 2018-11-07 15:14:30 -08:00
server mononoke: log reponame in wireproto logger 2018-11-06 03:03:25 -08:00
sshrelay rust/netstring: convert from error-chain to failure 2018-09-12 20:37:41 -07:00
storage Add Cargo.toml files to crates. (#7) 2018-07-09 19:52:27 -07:00
tests LFS verification tool 2018-11-08 03:41:40 -08:00
vfs mononoke: use err_downcast generally in mononoke 2018-09-06 14:24:08 -07:00
.gitignore add .gitignore 2018-03-13 11:58:20 -07:00
.travis.yml Add travis-CI integration by adding .travis.yml file. 2018-07-26 10:09:32 -07:00
Cargo.toml Restore cachelib blob caching 2018-08-07 11:37:37 -07:00
CONTRIBUTING.md Initial commit 2017-07-27 18:00:19 -07:00
LICENSE Initial commit 2017-07-27 18:00:19 -07:00
packman.yml packman config file for admin tool 2018-06-04 22:09:49 -07:00
README.md mention in README that Mononoke supports Mercurial 2017-07-28 14:41:37 -07:00
rustfmt.toml updates for rustfmt 0.3.4 2018-01-05 12:07:01 -08:00

Mononoke

Mononoke is a next-generation server for the Mercurial source control system, meant to scale up to accepting thousands of commits every hour across millions of files. It is primarily written in the Rust programming language.

Caveat Emptor

Mononoke is still in early stages of development. We are making it available now because we plan to start making references to it from our other open source projects such as Eden.

The version that we provide on GitHub does not build yet.

This is because the code is exported verbatim from an internal repository at Facebook, and not all of the scaffolding from our internal repository can be easily extracted. The key areas where we need to shore things up are:

  • Full support for a standard cargo build.
  • Open source replacements for Facebook-internal services (blob store, logging etc).

The current goal is to get Mononoke working on Linux. Other Unix-like OSes may be supported in the future.