facebook/sapling
1
1
Fork 0
mirror of https://github.com/facebook/sapling.git synced 2024-10-04 13:57:13 +03:00
Code Issues Projects Releases Wiki Activity

derivation_queue: wait for zelos watch when encountering an empty ready queue

Browse Source 
Summary:
When dequeuing items from the zelos derivation queue, first we query zelos for all nodes in the ready directory, and if the ready directory is empty, we will end up sleeping for 10 ms (current value of "scm/mononoke:derivation_worker_sleep_duration_ms") and then querying zelos again. For most repos this is excessive and puts a large load on zelos unnecessarily.
This diff instead makes use of Zelos' ability to set watches, by setting a watch on the ready queue when it's empty, and only querying zelos again when the watch returns a notification.

Differential Revision: D56939625

fbshipit-source-id: 6ac6c420b63c793ffd7b94d195f77173c4020a13
This commit is contained in:
Youssef Ibrahim 2024-05-03 14:51:58 -07:00 committed by Facebook GitHub Bot
parent be63c56e6b
commit 19926f39a6
2 changed files with 9 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
eden/mononoke/repo_attributes/repo_derivation_queues/src/errors.rs
View File

@ -15,8 +15,6 @@ use crate::DerivationDagItem;
#[derive(Error, Debug)]
pub enum InternalError {
#[error("Derivation queue is empty")]
EmptyQueue,
#[error("Got item for non-existing repo: {0}")]
RepoNotFound(RepositoryId),
#[error("Provided config_name: {0} was not found in the available configs")]

10
eden/mononoke/repo_attributes/repo_derivation_queues/src/lib.rs
View File

@ -51,7 +51,7 @@ pub trait DerivationQueue {
&self,
ctx: &CoreContext,
limit: usize,
) -> Result<Vec<DerivationDagItem>, InternalError>;
) -> Result<DequeueResponse, InternalError>;
async fn ack(&self, ctx: &CoreContext, item_id: DagItemId) -> Result<(), InternalError>;
@ -98,3 +98,11 @@ impl std::fmt::Debug for EnqueueResponse {
write!(f, "Wrapper for Zeus watch")
}
}
pub enum DequeueResponse {
Empty {
ready_queue_watch:
Box<dyn futures::future::Future<Output = anyhow::Result<()>> + Unpin + Send + Sync>,
},
Items(Vec<DerivationDagItem>),
}