Skip to content

fix(memory): reject pseudo tool outputs during extraction - #3733

Open
huangxun375-stack wants to merge 1 commit into
volcengine:mainfrom
huangxun375-stack:codex/fix-memory-extraction-protocol
Open

fix(memory): reject pseudo tool outputs during extraction#3733
huangxun375-stack wants to merge 1 commit into
volcengine:mainfrom
huangxun375-stack:codex/fix-memory-extraction-protocol

Conversation

@huangxun375-stack

Copy link
Copy Markdown
Contributor

Description

Memory extraction can receive a non-empty JSON object shaped like a tool request, for example {"tool_call_name":"search",...}. The parser currently drops unknown fields and validates the resulting empty object as a successful no-op, so a session commit can complete while silently extracting no memories.

This change aligns the extraction prompt and message envelope with the tools that are actually available, and rejects non-empty JSON objects that contain no recognized operation fields so the existing format-retry path can recover.

Human Involvement

  • A human participated in the implementation or review loop
  • This PR was generated entirely by AI agents without human participation in the loop

Related Issue

N/A

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

Changes Made

  • Make eager extraction explicitly tool-free and expose only read in the on-demand path.
  • Serialize pre-fetched results as message_type=tool_result instead of imitating a callable tool request.
  • Reject non-empty JSON objects with no recognized schema fields while preserving explicit {} and allowed [] no-op responses.

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this on the following platforms:
    • Linux
    • macOS
    • Windows

Focused and related checks:

  • 90 related memory tests passed.
  • Ruff passed for all changed Python files.
  • git diff --check origin/main...HEAD passed.

Direct OpenViking REST E2E, using identical LoCoMo conv-26 payloads across all 19 sessions with deepseek-v4-pro-260506:

Metric main@c7ee8e7 Fix b5a5914
Commit tasks completed 19/19 19/19
Completed sessions with zero memory operations 12/19 0/19
Total memory operations 52 144
Durable semantic memory files 42 111
Provider/task/extraction failures 0 0

The replay also verified message/archive fidelity, readable memory_diff.json files, and readable durable memories. No No memory files to vectorize signal occurred in either valid run.

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Screenshots (if applicable)

N/A

Additional Notes

The fix reuses the existing extraction format-retry state machine. It does not add a new retry mechanism or change the meaning of an explicit valid no-op response.

@chenjw

chenjw commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

locomo上的打分有变化么

@huangxun375-stack

huangxun375-stack commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

locomo上的打分有变化么

使用 DeepSeek-V4-pro

场景 LoCoMo case0 零记忆产出 session
未修首轮 71/152(46.71%) 7/19
修复后 117/152(76.97%) 0/19

@huangxun375-stack

Copy link
Copy Markdown
Contributor Author

locomo上的打分有变化么

和模型有关,DeepSeek-V4-pro会因为这个误导难以提取记忆,豆包和gpt没有出现。

@chenjw

chenjw commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

我先合入了 @fujiajie666 后面跑分的时候可以关注下,是否有变化

else ""
)
context_workflow = (
"2. Use only the pre-fetched context; no tools are available\n"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里好像有问题,模型并不知道pre-fetched的概念吧?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

2 participants