facebook/sapling
1
1
Fork 0
mirror of https://github.com/facebook/sapling.git synced 2024-12-29 08:02:24 +03:00
Code Issues Projects Releases Wiki Activity

live_commit_sync_config: make scs xrepo-lookup watch live changes

Browse Source 
Summary:
When we change `CommitSyncConfig`, we want to not have to restart `scs` servers, and instead have them pick up the new config by using `LiveCommitSyncConfig`.

This diff turned out larger than I expected, mainly due to the need to introduce various things around `TestLiveCommitSyncConfig`:
- `TestLiveCommitSyncConfig`: the trait implementer to use in `mononoke_api::Repo`
- `TestLiveCommitSyncConfigSource`: the helper struct to keep around for new values injection (very similar to how our `ConfigStore` has an inner `ConfigSource`, which can also be `TestSource`, but here injected values can be `CommitSyncConfig` instead of JSON).
- all the places in integration tests, where `setup_configerator_configs` is now needed (I plan to start setting up configerator configs always in a separate diff, as it is cheap)

Here are the explanations for a few things I think may be not immediately obvious:
- I removed the `Clone` bound from `LiveCommitSyncConfig` trait, as `Clone` traits [cannot be made into trait objects](https://doc.rust-lang.org/book/ch17-02-trait-objects.html#object-safety-is-required-for-trait-objects)
- `TestLiveCommitSyncConfigSource` vs `TestLiveCommitSyncConfigSourceInner` discrepancy is to ensure nobody should instantiate `TestLiveCommitSyncConfigSourceInner` outside of `live_commit_sync_config/src`
- I am aware of the ugly discrepancy between the main `--mononoke-config-path`, which is used to load initial configuration and can be both a file-based and a configerator-based config; and `--local-configerator-path`, used to override config sources for `Tunables` and `LiveCommitSyncConfig`. Ideally these two should just be unified somehow, but that is a little out of scope of this diff (I've already added it to the dirt doc though).
- in `mononoke_api::Repo` there are methods `new_test_xrepo` and `new_test_common`, which changed from maybe accepting just a `CommitSyncConfig` to now maybe accepting both `CommitSyncConfig` and `LiveCommitSyncConfig`. It can be made a bit cleaner: I can just query `CommitSyncConfig` from `LiveCommitSyncConfig` in `new_test_common` and avoid having two args. I was too lazy to do this, lmk if you feel strongly about it.

Reviewed By: StanislavGlebik

Differential Revision: D22443623

fbshipit-source-id: 0d6bbda2223e77b89cc59863b703db5081bcd601
This commit is contained in:
Kostia Balytskyi 2020-07-21 09:07:15 -07:00 committed by Facebook GitHub Bot
parent c5e880c239
commit fc61c74f23
24 changed files with 415 additions and 76 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
eden/mononoke/commit_rewriting/cross_repo_sync/test_utils/Cargo.toml
View File

@ -10,6 +10,7 @@ include = ["**/*.rs"]
path = "lib.rs"
[dependencies]
blobrepo = { path = "../../../blobrepo" }
blobrepo_factory = { path = "../../../blobrepo/factory" }
blobrepo_hg = { path = "../../../blobrepo/blobrepo_hg" }
blobstore = { path = "../../../blobstore" }

39
eden/mononoke/commit_rewriting/cross_repo_sync/test_utils/lib.rs
View File

@ -10,6 +10,7 @@
use ascii::AsciiString;
use anyhow::{format_err, Error};
use blobrepo::BlobRepo;
use blobrepo_hg::BlobRepoHg;
use blobstore::Loadable;
use bookmarks::{BookmarkName, BookmarkUpdateReason};
@ -249,32 +250,34 @@ pub async fn init_small_large_repo(
.await?
);
let small_repo_sync_config = SmallRepoCommitSyncConfig {
default_action: DefaultSmallToLargeCommitSyncPathAction::PrependPrefix(MPath::new(
"prefix",
)?),
map: hashmap! {},
bookmark_prefix: AsciiString::new(),
direction: CommitSyncDirection::SmallToLarge,
};
let commit_sync_config = CommitSyncConfig {
large_repo_id: megarepo.get_repoid(),
common_pushrebase_bookmarks: vec![],
small_repos: hashmap! {
smallrepo.get_repoid() => small_repo_sync_config,
},
version_name: CommitSyncConfigVersion("TEST_VERSION_NAME".to_string()),
};
Ok((
Syncers {
small_to_large: small_to_large_commit_syncer,
large_to_small: large_to_small_commit_syncer,
},
commit_sync_config,
base_commit_sync_config(&megarepo, &smallrepo),
))
}
pub fn base_commit_sync_config(large_repo: &BlobRepo, small_repo: &BlobRepo) -> CommitSyncConfig {
let small_repo_sync_config = SmallRepoCommitSyncConfig {
default_action: DefaultSmallToLargeCommitSyncPathAction::PrependPrefix(
MPath::new("prefix").unwrap(),
),
map: hashmap! {},
bookmark_prefix: AsciiString::new(),
direction: CommitSyncDirection::SmallToLarge,
};
CommitSyncConfig {
large_repo_id: large_repo.get_repoid(),
common_pushrebase_bookmarks: vec![],
small_repos: hashmap! {
small_repo.get_repoid() => small_repo_sync_config,
},
version_name: CommitSyncConfigVersion("TEST_VERSION_NAME".to_string()),
}
}
fn identity_renamer(b: &BookmarkName) -> Option<BookmarkName> {
Some(b.clone())
}

188
eden/mononoke/commit_rewriting/live_commit_sync_config/src/lib.rs
View File

@ -18,6 +18,7 @@ use pushredirect_enable::types::{MononokePushRedirectEnable, PushRedirectEnableS
use repos::types::RawCommitSyncConfig;
use slog::{debug, error, info, Logger};
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use thiserror::Error;
pub const CONFIGERATOR_PUSHREDIRECT_ENABLE: &str = "scm/mononoke/pushredirect/enable";
@ -41,7 +42,7 @@ pub enum ErrorKind {
UnknownCommitSyncConfigVersion(RepositoryId, String),
}
pub trait LiveCommitSyncConfig: Clone {
pub trait LiveCommitSyncConfig: Send + Sync {
/// Return whether push redirection is currently
/// enabled for draft commits in `repo_id`
///
@ -263,3 +264,188 @@ impl LiveCommitSyncConfig for CfgrLiveCommitSyncConfig {
})
}
}
/// Inner container for `TestLiveCommitSyncConfigSource`
/// See `TestLiveCommitSyncConfigSource` for more details
struct TestLiveCommitSyncConfigSourceInner {
commit_sync_config: Mutex<HashMap<RepositoryId, CommitSyncConfig>>,
push_redirection_for_draft: Mutex<HashMap<RepositoryId, bool>>,
push_redirection_for_public: Mutex<HashMap<RepositoryId, bool>>,
}
/// A helper type to manage `TestLiveCommitSyncConfig` from outside
/// The idea behind `TestLiveCommitSyncConfig` is that it is going
/// to be used in type-erased contexts, behind `dyn LiveCommitSyncConfig`.
/// Therefore there will be no way to access anything beyond the
/// `LiveCommitSyncConfig` interface, so no way to edit existing config.
/// To allow test scenarios to edit underlying configs, creators of
/// `TestLiveCommitSyncConfig` also receive an accompanying
/// `TestLiveCommitSyncConfigSource`, which allows editing underlying
/// configs
#[derive(Clone)]
pub struct TestLiveCommitSyncConfigSource(Arc<TestLiveCommitSyncConfigSourceInner>);
impl TestLiveCommitSyncConfigSource {
fn new() -> Self {
Self(Arc::new(TestLiveCommitSyncConfigSourceInner {
commit_sync_config: Mutex::new(HashMap::new()),
push_redirection_for_draft: Mutex::new(HashMap::new()),
push_redirection_for_public: Mutex::new(HashMap::new()),
}))
}
pub fn set_draft_push_redirection_enabled(&self, repo_id: RepositoryId) {
self.0
.push_redirection_for_draft
.lock()
.expect("poisoned lock")
.insert(repo_id, true);
}
pub fn set_public_push_redirection_enabled(&self, repo_id: RepositoryId) {
self.0
.push_redirection_for_public
.lock()
.expect("poisoned lock")
.insert(repo_id, true);
}
pub fn set_commit_sync_config(
&self,
repo_id: RepositoryId,
commit_sync_config: CommitSyncConfig,
) {
self.0
.commit_sync_config
.lock()
.expect("poisoned lock")
.insert(repo_id, commit_sync_config);
}
pub fn get_commit_sync_config_for_repo(
&self,
repo_id: RepositoryId,
) -> Result<CommitSyncConfig> {
self.0
.commit_sync_config
.lock()
.expect("poisoned lock")
.get(&repo_id)
.ok_or_else(|| ErrorKind::NotPartOfAnyCommitSyncConfig(repo_id).into())
.map(|v| v.clone())
}
fn push_redirector_enabled_for_draft(&self, repo_id: RepositoryId) -> bool {
*self
.0
.push_redirection_for_draft
.lock()
.expect("poisoned lock")
.get(&repo_id)
.unwrap_or(&false)
}
fn push_redirector_enabled_for_public(&self, repo_id: RepositoryId) -> bool {
*self
.0
.push_redirection_for_public
.lock()
.expect("poisoned lock")
.get(&repo_id)
.unwrap_or(&false)
}
fn get_current_commit_sync_config(
&self,
_ctx: &CoreContext,
repo_id: RepositoryId,
) -> Result<CommitSyncConfig> {
self.get_commit_sync_config_for_repo(repo_id)
}
fn get_all_commit_sync_config_versions(
&self,
repo_id: RepositoryId,
) -> Result<HashMap<CommitSyncConfigVersion, CommitSyncConfig>> {
let c = self.get_commit_sync_config_for_repo(repo_id)?;
let hm = {
let mut hm: HashMap<_, _> = HashMap::new();
hm.insert(c.version_name.clone(), c);
hm
};
Ok(hm)
}
fn get_commit_sync_config_by_version(
&self,
repo_id: RepositoryId,
version_name: &CommitSyncConfigVersion,
) -> Result<CommitSyncConfig> {
self.get_commit_sync_config_for_repo(repo_id).map_err(|_| {
ErrorKind::UnknownCommitSyncConfigVersion(repo_id, version_name.0.clone()).into()
})
}
}
/// A unit-test freindly implementor of `LiveCommitSyncConfig`
/// As this struct is meant to be held behind a type-erasing
/// `dyn LiveCommitSyncConfig`, anything beyond the interface
/// of `LiveCommitSyncConfig` won't be visible to the users.
/// Therefore, to modify internal state a `TestLiveCommitSyncConfigSource`
/// should be used.
#[derive(Clone)]
pub struct TestLiveCommitSyncConfig {
source: TestLiveCommitSyncConfigSource,
}
impl TestLiveCommitSyncConfig {
pub fn new_with_source() -> (Self, TestLiveCommitSyncConfigSource) {
let source = TestLiveCommitSyncConfigSource::new();
(
Self {
source: source.clone(),
},
source,
)
}
pub fn new_empty() -> Self {
let source = TestLiveCommitSyncConfigSource::new();
Self { source }
}
}
impl LiveCommitSyncConfig for TestLiveCommitSyncConfig {
fn push_redirector_enabled_for_draft(&self, repo_id: RepositoryId) -> bool {
self.source.push_redirector_enabled_for_draft(repo_id)
}
fn push_redirector_enabled_for_public(&self, repo_id: RepositoryId) -> bool {
self.source.push_redirector_enabled_for_public(repo_id)
}
fn get_current_commit_sync_config(
&self,
ctx: &CoreContext,
repo_id: RepositoryId,
) -> Result<CommitSyncConfig> {
self.source.get_current_commit_sync_config(ctx, repo_id)
}
fn get_all_commit_sync_config_versions(
&self,
repo_id: RepositoryId,
) -> Result<HashMap<CommitSyncConfigVersion, CommitSyncConfig>> {
self.source.get_all_commit_sync_config_versions(repo_id)
}
fn get_commit_sync_config_by_version(
&self,
repo_id: RepositoryId,
version_name: &CommitSyncConfigVersion,
) -> Result<CommitSyncConfig> {
self.source
.get_commit_sync_config_by_version(repo_id, version_name)
}
}

4
eden/mononoke/edenapi_server/src/main.rs
View File

@ -117,6 +117,8 @@ async fn start(
let blobstore_options = args::parse_blobstore_options(&matches);
let trusted_proxy_idents = parse_identities(&matches)?;
let tls_session_data_log = matches.value_of(ARG_TLS_SESSION_DATA_LOG_FILE);
let config_store = args::maybe_init_config_store(fb, &logger, &matches)
.expect("failed to instantiate ConfigStore");
debug!(logger, "Initializing Mononoke API");
let mononoke = Mononoke::new(
@ -127,6 +129,7 @@ async fn start(
caching,
readonly_storage,
blobstore_options,
config_store,
)
.await?;
@ -228,6 +231,7 @@ fn main(fb: FacebookInit) -> Result<()> {
.with_fb303_args()
.with_all_repos()
.with_shutdown_timeout_args()
.with_test_args()
.build()
.arg(
Arg::with_name(ARG_LISTEN_HOST)

2
eden/mononoke/mononoke_api/Cargo.toml
View File

@ -22,6 +22,7 @@ fastlog = { path = "../derived_data/fastlog" }
filestore = { path = "../filestore" }
fsnodes = { path = "../derived_data/fsnodes" }
hgproto = { path = "../hgproto" }
live_commit_sync_config = { path = "../commit_rewriting/live_commit_sync_config" }
manifest = { path = "../manifest" }
mercurial_types = { path = "../mercurial/types" }
metaconfig_parser = { path = "../metaconfig/parser" }
@ -40,6 +41,7 @@ unbundle = { path = "../repo_client/unbundle" }
unodes = { path = "../derived_data/unodes" }
warm_bookmarks_cache = { path = "../bookmarks/warm_bookmarks_cache" }
xdiff = { path = "../../scm/lib/xdiff" }
cached_config = { git = "https://github.com/facebookexperimental/rust-shed.git", branch = "master" }
cloned = { git = "https://github.com/facebookexperimental/rust-shed.git", branch = "master" }
fbinit = { git = "https://github.com/facebookexperimental/rust-shed.git", branch = "master" }
stats = { git = "https://github.com/facebookexperimental/rust-shed.git", branch = "master" }

77
eden/mononoke/mononoke_api/src/lib.rs
View File

@ -14,6 +14,7 @@ use std::sync::Arc;
use anyhow::Error;
use blobrepo_factory::{BlobstoreOptions, Caching, ReadOnlyStorage};
use cached_config::ConfigStore;
use cloned::cloned;
use fbinit::FacebookInit;
use futures::future;
@ -75,6 +76,7 @@ impl Mononoke {
with_cachelib: Caching,
readonly_storage: ReadOnlyStorage,
blobstore_options: BlobstoreOptions,
config_store: ConfigStore,
) -> Result<Self, Error> {
let common_config = configs.common;
let repos = future::join_all(
@ -82,25 +84,28 @@ impl Mononoke {
.repos
.into_iter()
.filter(move |&(_, ref config)| config.enabled)
.map(move |(name, config)| {
cloned!(logger, common_config, blobstore_options);
async move {
info!(logger, "Initializing repo: {}", &name);
let repo = Repo::new(
fb,
logger.new(o!("repo" => name.clone())),
name.clone(),
config,
common_config,
mysql_options,
with_cachelib,
readonly_storage,
blobstore_options,
)
.await
.expect("failed to initialize repo");
debug!(logger, "Initialized {}", &name);
(name, Arc::new(repo))
.map({
move |(name, config)| {
cloned!(logger, common_config, blobstore_options, config_store);
async move {
info!(logger, "Initializing repo: {}", &name);
let repo = Repo::new(
fb,
logger.new(o!("repo" => name.clone())),
name.clone(),
config,
common_config,
mysql_options,
with_cachelib,
readonly_storage,
blobstore_options,
config_store,
)
.await
.expect("failed to initialize repo");
debug!(logger, "Initialized {}", &name);
(name, Arc::new(repo))
}
}
}),
)
@ -142,6 +147,9 @@ impl Mononoke {
mod test_impl {
use super::*;
use blobrepo::BlobRepo;
use live_commit_sync_config::{
LiveCommitSyncConfig, TestLiveCommitSyncConfig, TestLiveCommitSyncConfigSource,
};
use metaconfig_types::CommitSyncConfig;
use synced_commit_mapping::SyncedCommitMapping;
@ -179,30 +187,33 @@ mod test_impl {
Arc<dyn SyncedCommitMapping>,
),
>,
) -> Result<Self, Error> {
) -> Result<(Self, TestLiveCommitSyncConfigSource), Error> {
use futures::stream::{FuturesOrdered, TryStreamExt};
let (lv_cfg, lv_cfg_src) = TestLiveCommitSyncConfig::new_with_source();
let lv_cfg: Arc<dyn LiveCommitSyncConfig> = Arc::new(lv_cfg);
let repos = repos
.into_iter()
.map(
.map({
cloned!(lv_cfg_src);
move |(name, repo, commit_sync_config, synced_commit_maping)| {
cloned!(ctx);
cloned!(ctx, lv_cfg, lv_cfg_src);
async move {
Repo::new_test_xrepo(
ctx.clone(),
repo,
commit_sync_config,
synced_commit_maping,
)
.await
.map(move |repo| (name, Arc::new(repo)))
lv_cfg_src.set_commit_sync_config(
repo.get_repoid(),
commit_sync_config.clone(),
);
Repo::new_test_xrepo(ctx.clone(), repo, lv_cfg, synced_commit_maping)
.await
.map(move |repo| (name, Arc::new(repo)))
}
},
)
}
})
.collect::<FuturesOrdered<_>>()
.try_collect()
.await?;
Ok(Self { repos })
Ok((Self { repos }, lv_cfg_src))
}
}
}

66
eden/mononoke/mononoke_api/src/repo.rs
View File

@ -21,6 +21,7 @@ use blobstore_factory::make_metadata_sql_factory;
use bookmarks::{
BookmarkKind, BookmarkName, BookmarkPagination, BookmarkPrefix, Bookmarks, Freshness,
};
use cached_config::ConfigStore;
use changeset_info::ChangesetInfo;
use context::CoreContext;
use cross_repo_sync::{CommitSyncRepos, CommitSyncer};
@ -33,9 +34,12 @@ use futures::stream::{StreamExt, TryStreamExt};
use futures::try_join;
use futures_old::stream::Stream;
use itertools::Itertools;
#[cfg(test)]
use live_commit_sync_config::TestLiveCommitSyncConfig;
use live_commit_sync_config::{CfgrLiveCommitSyncConfig, LiveCommitSyncConfig};
use mercurial_types::Globalrev;
#[cfg(test)]
use metaconfig_types::{CommitSyncConfig, SourceControlServiceParams};
use metaconfig_types::SourceControlServiceParams;
use metaconfig_types::{CommonConfig, RepoConfig};
use mononoke_types::{
hash::{GitSha1, Sha1, Sha256},
@ -92,6 +96,7 @@ pub(crate) struct Repo {
pub(crate) config: RepoConfig,
pub(crate) repo_permission_checker: ArcPermissionChecker,
pub(crate) service_permission_checker: ArcPermissionChecker,
pub(crate) live_commit_sync_config: Arc<dyn LiveCommitSyncConfig>,
}
#[derive(Clone)]
@ -142,6 +147,7 @@ impl Repo {
with_cachelib: Caching,
readonly_storage: ReadOnlyStorage,
blobstore_options: BlobstoreOptions,
config_store: ConfigStore,
) -> Result<Self, Error> {
let skiplist_index_blobstore_key = config.skiplist_index_blobstore_key.clone();
@ -154,6 +160,9 @@ impl Repo {
)
.await?;
let live_commit_sync_config: Arc<dyn LiveCommitSyncConfig> =
Arc::new(CfgrLiveCommitSyncConfig::new(&logger, &config_store)?);
let builder = BlobrepoBuilder::new(
fb,
name.clone(),
@ -217,6 +226,7 @@ impl Repo {
config,
repo_permission_checker,
service_permission_checker,
live_commit_sync_config,
})
}
@ -237,13 +247,13 @@ impl Repo {
pub(crate) async fn new_test_xrepo(
ctx: CoreContext,
blob_repo: BlobRepo,
commit_sync_config: CommitSyncConfig,
live_commit_sync_config: Arc<dyn LiveCommitSyncConfig>,
synced_commit_mapping: Arc<dyn SyncedCommitMapping>,
) -> Result<Self, Error> {
Self::new_test_common(
ctx,
blob_repo,
Some(commit_sync_config),
Some(live_commit_sync_config),
synced_commit_mapping,
)
.await
@ -254,11 +264,16 @@ impl Repo {
async fn new_test_common(
ctx: CoreContext,
blob_repo: BlobRepo,
commit_sync_config: Option<CommitSyncConfig>,
live_commit_sync_config: Option<Arc<dyn LiveCommitSyncConfig>>,
synced_commit_mapping: Arc<dyn SyncedCommitMapping>,
) -> Result<Self, Error> {
let init_commit_sync_config = live_commit_sync_config
.as_ref()
.map(|lcsc| lcsc.get_current_commit_sync_config(&ctx, blob_repo.get_repoid()))
.transpose()?;
let config = RepoConfig {
commit_sync_config,
commit_sync_config: init_commit_sync_config,
source_control_service: SourceControlServiceParams {
permit_writes: true,
..Default::default()
@ -270,6 +285,12 @@ impl Repo {
.compat()
.await?,
);
let live_commit_sync_config: Arc<dyn LiveCommitSyncConfig> = match live_commit_sync_config {
Some(live_commit_sync_config) => live_commit_sync_config,
None => Arc::new(TestLiveCommitSyncConfig::new_empty()),
};
Ok(Self {
name: String::from("test"),
blob_repo,
@ -283,6 +304,7 @@ impl Repo {
service_permission_checker: ArcPermissionChecker::from(
PermissionCheckerBuilder::always_allow(),
),
live_commit_sync_config,
})
}
@ -582,6 +604,11 @@ impl RepoContext {
&self.repo.blob_repo
}
/// `LiveCommitSyncConfig` instance to query current state of sync configs.
pub(crate) fn live_commit_sync_config(&self) -> Arc<dyn LiveCommitSyncConfig> {
self.repo.live_commit_sync_config.clone()
}
/// The skiplist index for the referenced repository.
pub(crate) fn skiplist_index(&self) -> &SkiplistIndex {
&self.repo.skiplist_index
@ -974,19 +1001,22 @@ impl RepoContext {
other: &Self,
specifier: ChangesetSpecifier,
) -> Result<Option<ChangesetContext>, MononokeError> {
let commit_sync_repos = match &self.config().commit_sync_config {
Some(commit_sync_config) => CommitSyncRepos::new(
self.blob_repo().clone(),
other.blob_repo().clone(),
&commit_sync_config,
)?,
None => {
return Err(MononokeError::InvalidRequest(format!(
"Commits from {} are not configured to be remapped to another repo",
self.repo.name
)));
}
};
let commit_sync_config = self
.live_commit_sync_config()
.get_current_commit_sync_config(self.ctx(), self.blob_repo().get_repoid())
.map_err(|e| {
MononokeError::InvalidRequest(format!(
"Commits from {} are not configured to be remapped to another repo: {}",
self.repo.name, e
))
})?;
let commit_sync_repos = CommitSyncRepos::new(
self.blob_repo().clone(),
other.blob_repo().clone(),
&commit_sync_config,
)?;
let changeset =
self.resolve_specifier(specifier)
.await?

88
eden/mononoke/mononoke_api/src/test/test_repo.rs
View File

@ -26,6 +26,8 @@ use crate::{
TreeEntry, TreeId,
};
use cross_repo_sync_test_utils::init_small_large_repo;
use live_commit_sync_config::TestLiveCommitSyncConfigSource;
use metaconfig_types::{CommitSyncConfigVersion, DefaultSmallToLargeCommitSyncPathAction};
use mononoke_types::{
hash::{GitSha1, RichGitSha1, Sha1, Sha256},
MPath,
@ -562,7 +564,7 @@ async fn file_contents(fb: FacebookInit) -> Result<(), Error> {
#[fbinit::compat_test]
async fn xrepo_commit_lookup_simple(fb: FacebookInit) -> Result<(), Error> {
let ctx = CoreContext::test_mock(fb);
let mononoke = init_x_repo(&ctx).await?;
let (mononoke, _cfg_src) = init_x_repo(&ctx).await?;
let smallrepo = mononoke
.repo(ctx.clone(), "smallrepo")
@ -602,7 +604,7 @@ async fn xrepo_commit_lookup_simple(fb: FacebookInit) -> Result<(), Error> {
#[fbinit::compat_test]
async fn xrepo_commit_lookup_draft(fb: FacebookInit) -> Result<(), Error> {
let ctx = CoreContext::test_mock(fb);
let mononoke = init_x_repo(&ctx).await?;
let (mononoke, _cfg_src) = init_x_repo(&ctx).await?;
let smallrepo = mononoke
.repo(ctx.clone(), "smallrepo")
@ -658,7 +660,7 @@ async fn xrepo_commit_lookup_draft(fb: FacebookInit) -> Result<(), Error> {
#[fbinit::compat_test]
async fn xrepo_commit_lookup_public(fb: FacebookInit) -> Result<(), Error> {
let ctx = CoreContext::test_mock(fb);
let mononoke = init_x_repo(&ctx).await?;
let (mononoke, _cfg_src) = init_x_repo(&ctx).await?;
let smallrepo = mononoke
.repo(ctx.clone(), "smallrepo")
@ -711,7 +713,85 @@ async fn xrepo_commit_lookup_public(fb: FacebookInit) -> Result<(), Error> {
Ok(())
}
async fn init_x_repo(ctx: &CoreContext) -> Result<Mononoke, Error> {
#[fbinit::compat_test]
async fn xrepo_commit_lookup_config_changing_live(fb: FacebookInit) -> Result<(), Error> {
let ctx = CoreContext::test_mock(fb);
let (mononoke, cfg_src) = init_x_repo(&ctx).await?;
let smallrepo = mononoke
.repo(ctx.clone(), "smallrepo")
.await?
.expect("repo exists");
let largerepo = mononoke
.repo(ctx.clone(), "largerepo")
.await?
.expect("repo exists");
let large_master_cs_id = resolve_cs_id(&ctx, largerepo.blob_repo(), "master").await?;
// Before config change
let first_large =
CreateCommitContext::new(&ctx, largerepo.blob_repo(), vec![large_master_cs_id])
.add_file("prefix/remapped_before", "content1")
.add_file("not_remapped", "content2")
.commit()
.await?;
let first_small = largerepo
.xrepo_commit_lookup(&smallrepo, ChangesetSpecifier::Bonsai(first_large))
.await?;
let file_changes: Vec<_> = first_small
.unwrap()
.id()
.load(ctx.clone(), smallrepo.blob_repo().blobstore())
.await?
.file_changes()
.map(|(path, _)| path)
.cloned()
.collect();
assert_eq!(file_changes, vec![MPath::new("remapped_before")?]);
// Config change: new config remaps prefix2 instead of prefix
let large_repo_id = largerepo.blob_repo().get_repoid();
let small_repo_id = smallrepo.blob_repo().get_repoid();
let mut cfg = cfg_src.get_commit_sync_config_for_repo(large_repo_id)?;
cfg.small_repos
.get_mut(&small_repo_id)
.unwrap()
.default_action =
DefaultSmallToLargeCommitSyncPathAction::PrependPrefix(MPath::new("prefix2").unwrap());
cfg.version_name = CommitSyncConfigVersion("TEST_VERSION_NAME_2".to_string());
cfg_src.set_commit_sync_config(small_repo_id, cfg.clone());
cfg_src.set_commit_sync_config(large_repo_id, cfg);
// After config change
let second_large =
CreateCommitContext::new(&ctx, largerepo.blob_repo(), vec![large_master_cs_id])
.add_file("prefix2/remapped_after", "content1")
.add_file("not_remapped", "content2")
.commit()
.await?;
let second_small = largerepo
.xrepo_commit_lookup(&smallrepo, ChangesetSpecifier::Bonsai(second_large))
.await?;
let file_changes: Vec<_> = second_small
.unwrap()
.id()
.load(ctx.clone(), smallrepo.blob_repo().blobstore())
.await?
.file_changes()
.map(|(path, _)| path)
.cloned()
.collect();
assert_eq!(file_changes, vec![MPath::new("remapped_after")?]);
Ok(())
}
async fn init_x_repo(
ctx: &CoreContext,
) -> Result<(Mononoke, TestLiveCommitSyncConfigSource), Error> {
let (syncers, commit_sync_config) = init_small_large_repo(&ctx).await?;
let small_to_large = syncers.small_to_large;

4
eden/mononoke/scs_server/src/main.rs
View File

@ -58,6 +58,7 @@ fn main(fb: FacebookInit) -> Result<(), Error> {
.with_all_repos()
.with_shutdown_timeout_args()
.with_scuba_logging_args()
.with_test_args()
.build()
.arg(
Arg::with_name(ARG_HOST)
@ -89,6 +90,8 @@ fn main(fb: FacebookInit) -> Result<(), Error> {
let exec = runtime.executor();
let repo_configs = load_repo_configs(fb, config_path)?;
let config_store = args::maybe_init_config_store(fb, &logger, &matches)
.expect("failed to instantiate ConfigStore");
let mut scuba_builder = args::get_scuba_sample_builder(fb, &matches)?;
@ -102,6 +105,7 @@ fn main(fb: FacebookInit) -> Result<(), Error> {
caching,
args::parse_readonly_storage(&matches),
args::parse_blobstore_options(&matches),
config_store,
))?);
let will_exit = Arc::new(AtomicBool::new(false));

6
eden/mononoke/tests/integration/library.sh
View File

@ -850,7 +850,7 @@ CONFIG
function register_hook {
hook_name="$1"
shift 1
EXTRA_CONFIG_DESCRIPTOR=""
if [[ $# -gt 0 ]]; then
@ -926,6 +926,8 @@ function start_and_wait_for_scs_server {
GLOG_minloglevel=5 "$SCS_SERVER" "$@" \
-p "$SCS_PORT" \
--mononoke-config-path "$TESTTMP/mononoke-config" \
--test-instance \
--local-configerator-path="$TESTTMP/configerator" \
"${COMMON_ARGS[@]}" >> "$TESTTMP/scs_server.out" 2>&1 &
export SCS_SERVER_PID=$!
echo "$SCS_SERVER_PID" >> "$DAEMON_PIDS"
@ -969,6 +971,8 @@ function start_edenapi_server {
--listen-host 127.0.0.1 \
--listen-port "$port" \
--mononoke-config-path "$TESTTMP/mononoke-config" \
--test-instance \
--local-configerator-path="$TESTTMP/configerator" \
--tls-ca "$TEST_CERTDIR/root-ca.crt" \
--tls-private-key "$TEST_CERTDIR/localhost.key" \
--tls-certificate "$TEST_CERTDIR/localhost.crt" \

1
eden/mononoke/tests/integration/test-edenapi-server-complete-trees.t
View File

@ -8,6 +8,7 @@
Set up local hgrc and Mononoke config.
$ setup_common_config
$ setup_configerator_configs
$ cd $TESTTMP
Initialize test repo.

1
eden/mononoke/tests/integration/test-edenapi-server-files.t
View File

@ -8,6 +8,7 @@
Set up local hgrc and Mononoke config.
$ setup_common_config
$ setup_configerator_configs
$ cd $TESTTMP
Initialize test repo.

2
eden/mononoke/tests/integration/test-edenapi-server-history.t
View File

@ -8,6 +8,7 @@
Set up local hgrc and Mononoke config.
$ setup_common_config
$ setup_configerator_configs
$ cd $TESTTMP
Initialize test repo.
@ -94,3 +95,4 @@ Check history content.
linknode: f91e155a86e1b909d99174818a2f98de2c128c59

1
eden/mononoke/tests/integration/test-edenapi-server-trees.t
View File

@ -8,6 +8,7 @@
Set up local hgrc and Mononoke config.
$ setup_common_config
$ setup_configerator_configs
$ cd $TESTTMP
Initialize test repo.

1
eden/mononoke/tests/integration/test-edenapi-server.t
View File

@ -8,6 +8,7 @@
Start up EdenAPI server.
$ setup_mononoke_config
$ setup_configerator_configs
$ start_edenapi_server
Hit health check endpoint.

1
eden/mononoke/tests/integration/test-scs-blame.t
View File

@ -8,6 +8,7 @@
Setup config repo:
$ setup_common_config
$ setup_configerator_configs
$ cd "$TESTTMP"
Setup testing repo for mononoke:

1
eden/mononoke/tests/integration/test-scs-common-base.t
View File

@ -8,6 +8,7 @@
Setup config repo:
$ setup_common_config
$ setup_configerator_configs
$ cd "$TESTTMP"
Setup testing repo for mononoke:

1
eden/mononoke/tests/integration/test-scs-diff.t
View File

@ -8,6 +8,7 @@
Setup config repo:
$ setup_common_config
$ setup_configerator_configs
$ cd "$TESTTMP"
Setup testing repo for mononoke:

2
eden/mononoke/tests/integration/test-scs-list-bookmarks.t
View File

@ -8,6 +8,7 @@
setup configuration
$ INFINITEPUSH_NAMESPACE_REGEX='^scratch/.+$' setup_common_config
$ setup_configerator_configs
$ cd "$TESTTMP"
Setup testing repo for mononoke:
@ -184,4 +185,3 @@ the --include-scratch option requires the prefix
$ scsc list-bookmarks -R repo --include-scratch
error: SourceControlService::repo_list_bookmarks failed with RequestError { kind: RequestErrorKind::INVALID_REQUEST, reason: "prefix required to list scratch bookmarks" }
[1]

1
eden/mononoke/tests/integration/test-scs-log.t
View File

@ -8,6 +8,7 @@
Setup config repo:
$ setup_common_config
$ setup_configerator_configs
$ cd "$TESTTMP"
Setup testing repo for mononoke:

1
eden/mononoke/tests/integration/test-scs-lookup.t
View File

@ -8,6 +8,7 @@
Setup config repo:
$ POPULATE_GIT_MAPPING=1 setup_common_config
$ setup_configerator_configs
$ cd "$TESTTMP"
Setup testing repo for mononoke:

1
eden/mononoke/tests/integration/test-scs-x-repo.t
View File

@ -9,6 +9,7 @@
Setup config repo:
$ cd "$TESTTMP"
$ setup_configerator_configs
$ create_large_small_repo
Setting up hg server repos
Blobimporting them

1
eden/mononoke/tests/integration/test-scs.t
View File

@ -8,6 +8,7 @@
Setup config repo:
$ POPULATE_GIT_MAPPING=1 setup_common_config
$ setup_configerator_configs
$ cd "$TESTTMP"
Setup testing repo for mononoke:

1
eden/scm/tests/test-edenapi-server.t
View File

@ -4,6 +4,7 @@ $ . "$TESTDIR/library.sh"
Start up EdenAPI server.
$ setup_mononoke_config
$ setup_configerator_configs
$ start_edenapi_server
Hit health check endpoint.