Skip to content

Fix stored tool conversation replay order#621

Open
gracjankubicki wants to merge 1 commit into
laravel:0.xfrom
gracjankubicki:fix/database-conversation-tool-replay-order
Open

Fix stored tool conversation replay order#621
gracjankubicki wants to merge 1 commit into
laravel:0.xfrom
gracjankubicki:fix/database-conversation-tool-replay-order

Conversation

@gracjankubicki

Copy link
Copy Markdown
Contributor

Summary

This PR fixes the replay order for stored assistant messages that contain tool calls, tool results, and final assistant content.

DatabaseConversationStore stores an assistant response as a single database row containing:

  • final assistant text in content
  • tool calls in tool_calls
  • tool results in tool_results

Before this change, that stored aggregate row was rehydrated as:

  1. an assistant message containing both tool_calls and final content
  2. a tool result message

That produces an invalid or misleading tool-calling history:

  1. the model requested a tool call
  2. the model already produced the final answer
  3. the application only then supplied the tool result

This PR rehydrates the stored row into the same logical order produced during live tool execution:

  1. assistant message with tool calls
  2. tool result message
  3. assistant message with the final text

The database storage format stays unchanged. Only the in-memory replay order returned by DatabaseConversationStore::getLatestConversationMessages() changes.

Why this order matters

This is not specific to one provider. The issue is in the provider-agnostic conversation store, and the rehydrated messages are later mapped into each provider's API format.

The correct flow for tool calling is consistently:

assistant tool call -> tool result -> assistant final answer

OpenAI

Docs:

Package code:

  • src/Gateway/OpenAi/Concerns/MapsMessages.php

OpenAI Responses API represents tool calls as function_call items and tool outputs as function_call_output items linked by call_id. Replaying final assistant text before function_call_output breaks the logical Responses API sequence.

Azure OpenAI

Docs:

Package code:

  • src/Gateway/AzureOpenAi/AzureOpenAiGateway.php

Azure OpenAI uses the same OpenAI message mapping concerns in this package, so it has the same replay-order requirement as OpenAI Responses-compatible flows.

xAI

Docs:

Package code:

  • src/Gateway/Xai/Concerns/MapsMessages.php

xAI Responses API also uses function_call_output linked by call_id. Tool output must be replayed before the final assistant text.

Anthropic

Docs:

Package code:

  • src/Gateway/Anthropic/Concerns/MapsMessages.php

Anthropic maps tool requests as tool_use blocks and tool outputs as tool_result blocks. The final assistant text should not be replayed in the same logical step as the tool_use when the corresponding tool_result has not yet appeared.

Gemini

Docs:

Package code:

  • src/Gateway/Gemini/Concerns/MapsMessages.php

Gemini uses functionCall and function response parts. Rehydrating the final text as a separate model message after the function response better matches the function-calling turn sequence.

Groq

Docs:

Package code:

  • src/Gateway/Groq/Concerns/MapsMessages.php

Groq follows a Chat Completions-like flow where an assistant message with tool_calls is followed by role: tool, and then by the final assistant response.

Mistral

Docs:

Package code:

  • src/Gateway/Mistral/Concerns/MapsMessages.php

Mistral documents the function-calling conversation flow as assistant function call, tool result, then assistant answer. This PR makes stored conversation replay match that order.

DeepSeek

Docs:

Package code:

  • src/Gateway/DeepSeek/Concerns/MapsMessages.php

DeepSeek uses a Chat Completions-like tool-calling format. Tool result messages should follow the assistant tool call and precede the final assistant answer.

OpenRouter

Docs:

Package code:

  • src/Gateway/OpenRouter/Concerns/MapsMessages.php

OpenRouter standardizes tool calling across models and providers using a Chat Completions-like interface. Replaying the stored history in tool-call order avoids passing a misleading message sequence to routed models.

Ollama

Docs:

Package code:

  • src/Gateway/Ollama/Concerns/MapsMessages.php

Ollama's tool-calling loop appends the model message with tool calls, appends the tool result, and then continues the loop until the model produces the final answer. This PR makes database replay follow that same order.

Amazon Bedrock

Docs:

Package code:

  • src/Gateway/Bedrock/BedrockTextGateway.php

Bedrock Converse API uses toolUse and toolResult, linked by toolUseId. A tool result is the response to a previous model tool request, so the final assistant response should be replayed after the toolResult.

Scope

This change intentionally keeps the PR small:

  • no database schema changes
  • no storage format changes
  • no public ConversationStore contract changes
  • no provider gateway changes
  • only stored conversation rehydration changes

Stored tool conversations now replay as:

AssistantMessage('', $toolCalls)
ToolResultMessage($toolResults) // when present
AssistantMessage($content) // when present

Tests

Added and updated storage tests to assert that stored tool conversations replay in the logical tool-calling order.

Local verification:

  • composer test -- --filter DatabaseConversationStoreTest
  • composer test:lint
  • composer test
  • git diff --check

Full local suite result: 1392 passed, 182 skipped due to missing external provider API keys.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant