git.iko.soy/crush
on main help sign in
fix: support local models with unknown max_tokens and context window (#2554)

Two fixes for local/custom model compatibility (LM Studio, Ollama, llama.cpp): 1. Don't send MaxOutputTokens when it's 0. Custom models not in the catwalk providers list have DefaultMaxTokens=0, which gets sent as max_tokens:0 in the API request. LM Studio rejects this with "maxPredictedTokens does not satisfy the schema". Fix: only send the field when the value is positive. 2. Skip auto-summarize when ContextWindow is 0. Custom models have ContextWindow=0, making remaining tokens negative, which immediately triggers summarize after the first response. The session resets with "previous session was interrupted because it got too long" even for short conversations. Fix: skip the check when context window is unknown. Fixes #1218 (regression), relates to #1583, #1591

commit741c18a21121fde99dbcadc7eadafca037ba790e authorKartik33 <Kartiktushir33@gmail.com> dateMon 6 Apr 2026, 06:23 (6 months ago) parent6d046c2
1 file changed+11−1
compare with main
unifiedsplit
internal/agent/agent.go+11−1
internal/agent/agent.go+11−1
@@ -249,12 +249,17 @@ func (a *sessionAgent) Run(ctx context.Context, call SessionAgentCall) (*fantasy
249249
250250 var currentAssistant *message.Message
251251 var shouldSummarize bool
252+ // Don't send MaxOutputTokens if 0 — some providers (e.g. LM Studio) reject it
253+ var maxOutputTokens *int64
254+ if call.MaxOutputTokens > 0 {
255+ maxOutputTokens = &call.MaxOutputTokens
256+ }
252257 result, err := agent.Stream(genCtx, fantasy.AgentStreamCall{
253258 Prompt: message.PromptWithTextAttachments(call.Prompt, call.Attachments),
254259 Files: files,
255260 Messages: history,
256261 ProviderOptions: call.ProviderOptions,
257− MaxOutputTokens: &call.MaxOutputTokens,
262+ MaxOutputTokens: maxOutputTokens,
258263 TopP: call.TopP,
259264 Temperature: call.Temperature,
260265 PresencePenalty: call.PresencePenalty,
@@ -425,6 +430,11 @@ func (a *sessionAgent) Run(ctx context.Context, call SessionAgentCall) (*fantasy
425430 StopWhen: []fantasy.StopCondition{
426431 func(_ []fantasy.StepResult) bool {
427432 cw := int64(largeModel.CatwalkCfg.ContextWindow)
433+ // If context window is unknown (0), skip auto-summarize
434+ // to avoid immediately truncating custom/local models.
435+ if cw == 0 {
436+ return false
437+ }
428438 tokens := currentSession.CompletionTokens + currentSession.PromptTokens
429439 remaining := cw - tokens
430440 var threshold int64