fix(chat): strip media-offload placeholder text from image history - #302
Open
chujieHong wants to merge 1 commit into
Open
fix(chat): strip media-offload placeholder text from image history#302chujieHong wants to merge 1 commit into
chujieHong wants to merge 1 commit into
Conversation
The MediaOffloadMiddleware writes a '[image offloaded: sha=… path=… size=…B mime=…; use read_file to retrieve bytes]' placeholder text block into LangGraph checkpoint state on every turn after the first one. That text is meant for the LLM to read_file the bytes back, not for the dashboard UI — after leaving and re-entering a chat, the user's image showed this internal text underneath it. On history serialization, strip the offload placeholder and the LLM-only 'User sent an image.' sentinel for user messages that carry an image in octop_inbound_attachments (the original image is rendered from there). The user's own caption is preserved. Pure-image user messages keep their entry (with empty content) so the dashboard still renders the attachment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
修复聊天记录里用户图片下方多出内部占位文本的 bug。
用户发送较大(>4KiB)的 inline 图片后,harness 的
MediaOffloadMiddleware(harness_agent/middleware/media_offload.py,非 octop 源码)会把图片字节 offload 到工作区.media-cache/<sha>.<ext>,并从第二轮起把对应 inline 块替换成一段占位文本:[image offloaded: sha=… path=… size=…B mime=…; use read_file to retrieve bytes]这段文本是写给 LLM 看的(提示模型用
read_file取回字节),会经 state 合并写回 LangGraph checkpoint。用户切出聊天再切回后,历史接口GET /threads/{tid}/history从 checkpoint 原样读出并序列化 → 前端在图片下方渲染出这段内部文本,属于 UX 缺陷。修复(
src/octop/api/routers/chat/serialize.py):_is_offload_placeholder_block,用正则^\s*\[(?:image|audio)\s+offloaded\s*:识别占位块;_serialize_history_message(msg, *, user=None):当消息 role=user 且additional_kwargs里带图片inbound_attachments(键octop_inbound_attachments)时,用_strip_image_only_text_blocks过滤掉占位文本和仅给 LLM 看的本地化哨兵文本(attachment_empty_image:用户发送了图片。/User sent an image.),用户自己写的 caption 原样保留;inbound_attachments(workspace_path → agentAttachmentAccessUrl → ChatAttachment)渲染原图;_load_thread_messages把user传入(locale 用于匹配本地化哨兵文本,User.locale默认zh);_strip_image_only_text_blocks直接调用等边界。Target branch
develop(feature / fix — default)main(release/*orhotfix/*only)Type of change
Test plan
pytest tests/unit/api/test_chat_polish.py全绿(含新增 8 例占位文本过滤用例)。[image offloaded: sha=…]占位文本。inbound_attachments的旧历史消息序列化行为不变,占位文本不会被误删。make allpasses locallyChecklist
CHANGELOG.md(if user-facing)