Ollama keep_alive on a loaded model cancels eviction and stalls queued requests
With OLLAMA_MAX_LOADED_MODELS=1, a request carrying keep_alive for the loaded model cancels its pending unload, so a request for another model waits out the entire keep_alive. On Ollama 0.40.8 the second model answered in 9 seconds without keep_alive but did not answer within 150 seconds with keep_alive 30m; sending keep_alive: 0 released the queue in 1.5 seconds. A fix is proposed in PR #19190 and is not yet released.
- Without keep_alive the second request answered in 9 s; with keep_alive 30m it did not answer within 150 s
- keep_alive: 0 on the loaded model released the waiting request in 1.5 s
- No errors surface: /api/ps shows a healthy model and scheduler messages are debug-level only
- The fix in PR #19190 is not included in v0.40.9-rc1
Read next
Software