I tried to include, e.g.,
options:
chat_template_kwargs:
enable_thinking: false
in a template as a way of controlling thinking behavior for certain templates, only to be told that "Extra inputs are not permitted". chat_template_kwargs can be passed to a llama.cpp endpoint and works as expected from other front-ends such as OpenWebUI.
On one hand, I would expect this program to blindly pass whatever key-value pairs are passed as options to the api request (maybe with a flag?). But even if that's not desirable, it seems to make sense to support chat_template_kwargs to control various behaviors of models for a specific prompt or template (e.g., enable_thinking or reasoning_effort, etc.)
I tried to include, e.g.,
in a template as a way of controlling thinking behavior for certain templates, only to be told that "Extra inputs are not permitted".
chat_template_kwargscan be passed to a llama.cpp endpoint and works as expected from other front-ends such as OpenWebUI.On one hand, I would expect this program to blindly pass whatever key-value pairs are passed as options to the api request (maybe with a flag?). But even if that's not desirable, it seems to make sense to support
chat_template_kwargsto control various behaviors of models for a specific prompt or template (e.g.,enable_thinkingorreasoning_effort, etc.)