Skip to content

fix(chat): strip media-offload placeholder text from image history - #302

Open
chujieHong wants to merge 1 commit into
TencentCloud:developfrom
chujieHong:fix/chat-history-offload-placeholder
Open

fix(chat): strip media-offload placeholder text from image history#302
chujieHong wants to merge 1 commit into
TencentCloud:developfrom
chujieHong:fix/chat-history-offload-placeholder

Conversation

@chujieHong

@chujieHong chujieHong commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

修复聊天记录里用户图片下方多出内部占位文本的 bug。

用户发送较大(>4KiB)的 inline 图片后,harness 的 MediaOffloadMiddlewareharness_agent/middleware/media_offload.py,非 octop 源码)会把图片字节 offload 到工作区 .media-cache/<sha>.<ext>,并从第二轮起把对应 inline 块替换成一段占位文本:

[image offloaded: sha=… path=… size=…B mime=…; use read_file to retrieve bytes]

这段文本是写给 LLM 看的(提示模型用 read_file 取回字节),会经 state 合并写回 LangGraph checkpoint。用户切出聊天再切回后,历史接口 GET /threads/{tid}/history 从 checkpoint 原样读出并序列化 → 前端在图片下方渲染出这段内部文本,属于 UX 缺陷。

修复src/octop/api/routers/chat/serialize.py):

  • 新增 _is_offload_placeholder_block,用正则 ^\s*\[(?:image|audio)\s+offloaded\s*: 识别占位块;
  • _serialize_history_message(msg, *, user=None):当消息 role=user 且 additional_kwargs 里带图片 inbound_attachments(键 octop_inbound_attachments)时,用 _strip_image_only_text_blocks 过滤掉占位文本和仅给 LLM 看的本地化哨兵文本(attachment_empty_image用户发送了图片。 / User sent an image.),用户自己写的 caption 原样保留;
  • 关键边界:过滤后 blocks 为空但消息带 inbound_attachments 时不能 return None——否则前端连图片都看不到。entry 保留空 content,前端从 inbound_attachments(workspace_path → agentAttachmentAccessUrl → ChatAttachment)渲染原图;
  • 调用方 _load_thread_messagesuser 传入(locale 用于匹配本地化哨兵文本,User.locale 默认 zh);
  • 新增 8 个单测,覆盖占位块匹配/过滤、caption 保留、无附件时保留占位文本、en locale、无 user 默认 zh、非图片附件不触发过滤、_strip_image_only_text_blocks 直接调用等边界。

Target branch

  • Base is develop (feature / fix — default)
  • Base is main (release/* or hotfix/* only)

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Refactor / chore
  • Release / hotfix

Test plan

  1. pytest tests/unit/api/test_chat_polish.py 全绿(含新增 8 例占位文本过滤用例)。
  2. 手工复现:在聊天里发一张 >4KiB 的图片 → 正常对话几轮 → 切出再切回 → 图片正常渲染,下方不再出现 [image offloaded: sha=…] 占位文本。
  3. 带 caption 场景:图片附带文字说明时,过滤后只保留用户文字,占位文本被移除。
  4. 纯图片消息:只发图不带文字,切回后图片依然显示(content 为空但 entry 不丢,由 inbound_attachments 渲染)。
  5. 非图片附件:语音 / 文件等非图片附件历史内容不受影响,原样保留。
  6. 兼容性:无 inbound_attachments 的旧历史消息序列化行为不变,占位文本不会被误删。
  • make all passes locally
  • Added/updated tests

Checklist

  • Updated CHANGELOG.md (if user-facing)
  • README / docs updated (if needed)

The MediaOffloadMiddleware writes a '[image offloaded: sha=… path=… size=…B
mime=…; use read_file to retrieve bytes]' placeholder text block into
LangGraph checkpoint state on every turn after the first one. That text is
meant for the LLM to read_file the bytes back, not for the dashboard UI —
after leaving and re-entering a chat, the user's image showed this internal
text underneath it.

On history serialization, strip the offload placeholder and the LLM-only
'User sent an image.' sentinel for user messages that carry an image in
octop_inbound_attachments (the original image is rendered from there). The
user's own caption is preserved. Pure-image user messages keep their entry
(with empty content) so the dashboard still renders the attachment.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant