Deep refactoring of streaming area

[igordayen]

OVERVIEW

Continuation of PR #1581 that sets foundation for vendor-agnostic streaming.

This PR focuses on deep refactoring of the streaming framework by partitioning it into Spring AI-dependent artifacts and vendor-agnostic ones.
The vendor-agnostic part relies on LlmMessageStreamer abstraction introduced in first PR, while the Spring AI part employs SpringAiLlmMessageStreamer.

Key Changes

• LlmService enhanced with streaming capability detection (supportsStreaming(), createMessageStreamer())
• StreamingLlmOperations moved to core.internal.streaming package alongside LlmOperations
• StreamingCapabilityDetector moved from api to spi (not exposed for user consumption). Split into vendor-agnostic caching layer and Spring AI-specific StreamingCapabilityVerifier
• StreamingChatClientOperations deprecated (fallback for legacy streaming config), functionality split into vendor-agnostic StreamingLlmOperationsImpl and Spring AI-dependent
  SpringAiLlmMessageStreamer
• Common message builders refactored into messagePromptBuilders.kt per first PR review (@jasperblues comments)

Execution Flow

AbstractLlmOperations implements StreamingLlmOperationsFactory interface (createStreamingOperations() API), materialized in OperationContextDelegate. Most code is vendor-agnostic;
Spring AI streaming invoked via SpringAiLlmService.

About Tools Invocation

Next, thirsd, PR will address gap in current streaming - Inspector-style callbacks similar to non-streaming.
Streaming is inherently event-driven.
Current approach opts for vendor-managed tool invocation: tools are embedded inside chunks, results included in conversation history, and stream continues transparently. This differs from
ToolLoop where tool results yield another LLM invocation within the loop.

Plugging others than Spring AI Message Streamer - sample

 /**                                                                                                                                                                                         
   * LangChain4j implementation of [LlmService].                                                                                                                                              
   */                                                                                                                                                                                         
  data class LangChain4jLlmService(                                                                                                                                                           
      override val name: String,                                                                                                                                                              
      override val provider: String,                                                                                                                                                          
      private val chatModel: ChatLanguageModel,                                                                                                                                               
      private val streamingModel: StreamingChatLanguageModel? = null,                                                                                                                         
      // ... other properties                                                                                                                                                                 
  ) : LlmService<ChatLanguageModel> {                                                                                                                                                         
                                                                                                                                                                                              
      override fun supportsStreaming(): Boolean = streamingModel != null                                                                                                                      
                                                                                                                                                                                              
      override fun createMessageStreamer(options: LlmOptions): LlmMessageStreamer {                                                                                                           
          val model = streamingModel                                                                                                                                                          
              ?: throw UnsupportedOperationException(                                                                                                                                         
                  "LangChain4j model '$name' does not have a streaming model configured"                                                                                                      
              )                                                                                                                                                                               
          return LangChain4jMessageStreamer(model)                                                                                                                                            
      }                                                                                                                                                                                       
                                                                                                                                                                                              
      // ... other LlmService methods                                                                                                                                                         
  }  

[igordayen]

Code duplication reported on Deprecated class, which will be eventually removed anyway.

[igordayen]

@jorander - for your awarness as you work on streaming testability. thank you

[jorander]

@igordayen
Moving creation of StreamingLlmOperations implementations into StreamingLlmOperationsFactory seems like a good move. I think that will enable creation of a combined interface (LlmOperations + StreamingLlmOperations + StreamingLlmOperationsFactory) for mocking in EmbabelMockitoIntegrationTest.

[sonarqubecloud]

Quality Gate failed

Failed conditions
11.7% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

[jorander]

Looks great!

@igordayen When this PR is merged I will update #1598 to take advantage of the changes done here.