Chat flow integration with LLM module

[ruwinirathnamalala]

This pull request introduces support for LLM orchestration streaming in the chatbot backend, integrating a new streaming workflow for LLM-generated responses. The main changes include adding a new HTTP endpoint for LLM streaming, implementing orchestration logic to handle streaming requests, and updating the workflow to conditionally route messages through the LLM orchestrator based on configuration. Additionally, environment variables and configuration files are updated to support the new orchestrator service.

LLM Orchestration Streaming Integration:

• Added a new /channels/:channelId/llm-stream HTTP endpoint in notification-server/src/server.js to trigger LLM orchestration streaming for a given channel, accepting chat history and message payloads.
• Implemented the createLLMOrchestrationStreamRequest function in notification-server/src/openSearch.js, which handles streaming requests to the LLM orchestrator, manages active connections, processes SSE responses, and sends stream events to clients.
• Updated exports in notification-server/src/openSearch.js and imports in notification-server/src/server.js to include the new orchestration streaming function. [1] [2]

Workflow and Queue Handling Updates:

• Enhanced the bot activity workflow in DSL/Ruuter.public/backoffice/POST/internal/message-to-bot.yml to check if the LLM module is active, fetch chat history, and route messages through the LLM stream endpoint if enabled.
• Modified notification-server/src/sseUtil.js to process pending stream requests by conditionally invoking the LLM orchestration stream logic if the request includes a message, otherwise falling back to the Azure OpenAI stream. [1] [2]

Configuration Updates:

• Added LLM_ORCHESTRATOR_URL to constants.ini and notification-server/.env to configure the LLM orchestration service endpoint. [1] [2]

[sonarqubecloud]

Quality Gate failed

Failed conditions
3 Security Hotspots
C Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE