MAIT-205: Replace ROAT Filter Function with OWUI Workspace Tool#48
MAIT-205: Replace ROAT Filter Function with OWUI Workspace Tool#48
Conversation
Add tools/roat_retrieval.py — a Workspace Tool that queries the ROAT knowledge base on-demand, letting the LLM decide when retrieval is needed instead of running on every user message. Disable functions/function.py by default (enabled: false header + ENABLE_CUSTOM_RAG_SERVICE env default changed to false) to prevent double context injection when the tool is active. Also apply minor linter fixes to the filter file (import order, type hint syntax). Update README to replace the filter setup blurb with step-by-step Tool activation instructions (import, valves, model config, function calling mode, per-chat activation).
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe changes migrate RAG integration from a legacy custom filter function to a new workspace tool-based retrieval mechanism. The legacy filter is disabled by default, the README documents explicit ROAT tool activation steps, and a new Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Tools as Tools.search_knowledge_base
participant ROAT as ROAT API Endpoint
participant Formatter as Response Formatter
Client->>Tools: search_knowledge_base(query)
Tools->>Tools: Validate rag_service_url config
Tools->>Tools: Emit progress event
Tools->>ROAT: POST /query (JSON: query, top_k, metadata_filters)
ROAT-->>Tools: HTTP 200 + relevance chunks
Tools->>Formatter: Parse chunks into {score, text}
Tools->>Formatter: Extract filenames & source metadata
Tools->>Tools: Emit per-source status events
Formatter-->>Tools: Formatted markdown context + sources
Tools-->>Client: Combined context string
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 0/1 reviews remaining, refill in 46 minutes.Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
functions/function.py (1)
135-136: ⚡ Quick winAlign
get_filename_from_extrasreturn type with actual behavior.At Line 136, the function can return
None, but it is annotated asstr. Please update the annotation to avoid incorrect type assumptions.Suggested diff
-def get_filename_from_extras(self, extras: dict) -> str: +def get_filename_from_extras(self, extras: dict) -> str | None: return extras.get("key") or extras.get("filename") or extras.get("name") or None🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@functions/function.py` around lines 135 - 136, The get_filename_from_extras function is annotated to return str but can return None; update its signature to return Optional[str] (import Optional from typing) so the type matches behavior and adjust any callers if they assume a non-None return; reference the get_filename_from_extras method to locate where to change the annotation and add the necessary import.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tools/roat_retrieval.py`:
- Around line 59-60: Normalize the extras variable to always be a dict by
replacing extras = ref.get("extras", {}) with extras = ref.get("extras") or {}
so None becomes {} and cannot raise AttributeError; then update any subsequent
usage sites that call extras.items() or extras.get() (e.g., the
metadata.extend(...) and extras.get(...) usages) to rely on this normalized dict
and remove protective checks, ensuring functions/methods that process "extras"
(the local extras variable and the metadata handling logic) operate safely.
- Line 40: Current logging call log.info("Calling ROAT for query: %s",
query[:80]) exposes raw user input; change it to log only metadata such as the
query length and a request or correlation id (e.g., use len(query) and
request_id/correlation_id if available) and remove the query content. Replace
the format string to something like "Calling ROAT: query_length=%d
request_id=%s" and pass len(query) and the existing request id (or generate one
earlier in the request handling code) so no PII is written to logs while
retaining traceability.
- Around line 137-183: The async method search_knowledge_base calls the blocking
synchronous helper _call_rag_service (which uses requests.post) and thus blocks
the event loop; fix this by invoking _call_rag_service inside asyncio.to_thread
and awaiting it (e.g., rag_result = await asyncio.to_thread(_call_rag_service,
self.valves.rag_service_url, self.valves.rag_service_api_key,
self.valves.rag_service_timeout, self.valves.top_k, query)), and ensure asyncio
is imported; keep the existing emit/error handling and return behavior
unchanged.
---
Nitpick comments:
In `@functions/function.py`:
- Around line 135-136: The get_filename_from_extras function is annotated to
return str but can return None; update its signature to return Optional[str]
(import Optional from typing) so the type matches behavior and adjust any
callers if they assume a non-None return; reference the get_filename_from_extras
method to locate where to change the annotation and add the necessary import.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 1dc34ad5-fbd6-4f01-a368-70b3f6b0273a
📒 Files selected for processing (3)
README.mdfunctions/function.pytools/roat_retrieval.py
- Normalize extras to dict safely with `or {}` to handle explicit null
- Remove query content from logs, log query length only
- Run blocking requests.post in asyncio.to_thread to avoid blocking the event loop
| # Read configuration from environment variables | ||
| DEFAULT_ENABLED = os.getenv("ENABLE_CUSTOM_RAG_SERVICE", "true").lower() == "true" | ||
| # Disabled by default — retrieval has moved to the ROAT Knowledge Base Search workspace tool (tools/roat_retrieval.py) | ||
| DEFAULT_ENABLED = os.getenv("ENABLE_CUSTOM_RAG_SERVICE", "false").lower() == "true" |
There was a problem hiding this comment.
We can keep other improvements made to this function code, but let's please revert the change that changes the DEFAULT_ENABLED. The state of the filter function is to be controlled via the OWUI interface, not via Valve's defaults
|
|
||
| The filter function that's responsible for the RAG service communication will also be | ||
| automatically provisioned and enabled globally. You can change these settings at the Admin panel | ||
| The filter function that was previously responsible for RAG service communication has been replaced |
There was a problem hiding this comment.
I think this fits more to be included in a changelog (what we don't yet have) rather than in a README. Let's instead simply ensure both the Filter function and the Tool are documented
|
|
||
| ### Activating the ROAT Tool | ||
|
|
||
| After first boot, manually register the tool: |
There was a problem hiding this comment.
Moving forward, we're going to remove the code that automatically enables the Filter function and replace it with the code that automatically enables the Tool instead. So these instructions on manual activation should be unnecessary
| Use this tool when: | ||
| - The user asks a factual question that likely requires domain-specific knowledge | ||
| - The user asks about documented processes, policies, data, or internal content | ||
| - The user's question cannot be confidently answered from general training knowledge alone |
There was a problem hiding this comment.
In fact, we want this tool used for most of the queries, and we want to disincentivize the LLM from using general knowledge. Thus, I propose to drop this line and instead replace it with something that would instruct the model to use this tool more frequently
We're of course just guessing for now until we have some evaluations in place
| Do NOT use this tool for: | ||
| - Casual greetings or small talk | ||
| - Simple arithmetic or general world knowledge | ||
| - Follow-up questions within an ongoing conversation where context was already retrieved |
There was a problem hiding this comment.
I don't think this specific instruction as follow up questions may require queries into the ROAT, let's try to reformulate this
| query: A concise search query derived from the user's question | ||
|
|
||
| Returns: | ||
| Retrieved context passages from the knowledge base, or a message that nothing was found. |
There was a problem hiding this comment.
Please reword into Retrieved documents chunks and their metadata.....
|
|
||
| source_obj = { | ||
| "source": {"name": source_name}, | ||
| "document": [text[:1000] if len(text) > 1000 else text], |
There was a problem hiding this comment.
While we're at it, can we please make this configurable via Valve, making it unlimited by default?
| { | ||
| "source": source_name, | ||
| "file": filename, | ||
| "relevance_score": score, |
There was a problem hiding this comment.
Not something new - it was like that in the Filter function, but I noticed that the relevance_score is not displayed by the OWUI when viewing the citation source. Do you have any ideas why?
| @@ -0,0 +1,203 @@ | |||
| """ | |||
| title: ROAT Knowledge Base Search | |||
There was a problem hiding this comment.
Please rename this to just the "Knowledge Base Search"
2 participants
Summary
tools/roat_retrieval.py— OWUI Workspace Tool that queries ROAT only when the LLM decides it's relevant, replacing the Filter that ran on every messagefunctions/function.pyby default (enabled: false+ env default changed tofalse) to prevent double context injection