zed-industries/zed
1
1
Fork 0
mirror of https://github.com/zed-industries/zed.git synced 2024-09-19 02:17:35 +03:00
Code Issues Packages Projects Releases Wiki Activity

Pass through Anthropic cache configuration when using Zed provider (#16685)

Browse Source 
This PR makes it so the model's cache configuration gets passed through
from the base model when using the Zed provider.

Release Notes:

- Fixed caching for Anthropic models when using the Zed provider.
This commit is contained in:
Marshall Bowers 2024-08-22 12:48:47 -04:00 committed by GitHub
parent 25cdd2ad25
commit 93642c9c51
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 15 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
crates/language_model/src/provider/cloud.rs
View File

@ -445,6 +445,21 @@ impl LanguageModel for CloudLanguageModel {
self.model.max_token_count()
}
fn cache_configuration(&self) -> Option<LanguageModelCacheConfiguration> {
match &self.model {
CloudModel::Anthropic(model) => {
model
.cache_configuration()
.map(|cache| LanguageModelCacheConfiguration {
max_cache_anchors: cache.max_cache_anchors,
should_speculate: cache.should_speculate,
min_total_token: cache.min_total_token,
})
}
CloudModel::OpenAi(_) | CloudModel::Google(_) | CloudModel::Zed(_) => None,
}
}
fn count_tokens(
&self,
request: LanguageModelRequest,