Feat/vllm 0.20 upgrade#46

Merged

alez007 merged 9 commits intomainfrom

feat/vllm-0.20-upgrade

Apr 28, 2026

Commits on Apr 27, 2026

feat: upgrade vLLM to 0.20.0 and harden inference loaders
Alex M
committed
feat: add flatten_message_content utility and integrate into llama_cpp
Alex M
committed

Commits on Apr 28, 2026

refactor: split llama_cpp into per-surface OpenAI serving handlers
Alex M
committed
fix: tag REQUEST_TOTAL by outcome instead of marking every request processed
Alex M
committed
fix: type llama_cpp stream iterator so pyright accepts run_in_executor
Alex M
committed
refactor: detect capabilities and emit OpenAI-compliant chat responses for transformers loader
Alex M
committed
fix(vllm): remove non-existent io_processor argument from OpenAIServingRender
Alex M
committed
fix(api): handle direct Response chunks and improve vLLM embedding error conversion
Alex M
committed
feat(vllm): add max_num_batched_tokens to VllmEngineConfig
Alex M
committed