ImageBlock with data URI prefix causes double-wrapping in OpenAI message conversion

[markstur]

ImageBlock with data URI prefix causes double-wrapping in OpenAI message conversion

Labels

bug, backend:openai, priority:medium

Description

Summary

When an ImageBlock is created with a full data URI prefix (data:image/png;base64,), the message_to_openai_message() helper function unconditionally adds another prefix, resulting in malformed image URLs that fail to render in OpenAI-compatible backends.

Expected Behavior

ImageBlock should accept base64 image data with or without the data URI prefix, and message_to_openai_message() should produce a valid OpenAI message with exactly one data:image/png;base64, prefix in the image URL.

Actual Behavior

When ImageBlock contains a data URI prefix, the conversion function adds a second prefix, producing:

data:image/png;base64,data:image/png;base64,iVBORw0KGgo...

This causes images to fail rendering in OpenAI API calls and other OpenAI-compatible backends.

Root Cause

The message_to_openai_message() function in mellea/helpers/openai_compatible_helpers.py unconditionally wraps all image values with the data URI prefix without checking if one already exists.

Steps to Reproduce

from mellea.backends.openai import OpenAIBackend
from mellea.core.base import ImageBlock
from mellea.stdlib.components import Message
from mellea.helpers.openai_compatible_helpers import message_to_openai_message

# Create ImageBlock with data URI prefix (valid per ImageBlock.is_valid_base64_png)
img = ImageBlock("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==")
msg = Message(role="user", content="Describe this image", images=[img])

# Convert to OpenAI format
openai_msg = message_to_openai_message(msg)
print(openai_msg["content"][1]["image_url"]["url"])
# Output: data:image/png;base64,data:image/png;base64,iVBORw0KGgo...
#         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ DOUBLE PREFIX

Impact

• Images fail to render when using OpenAI, Azure OpenAI, or other OpenAI-compatible backends
• Affects any code that creates ImageBlock instances with the full data URI (which is explicitly supported by ImageBlock.is_valid_base64_png())
• Silent failure - no error is raised, images just don't display

Additional Context

The ImageBlock class explicitly supports both formats in its validation:

# From mellea/core/base.py lines 121-123
if "data:" in s and "base64," in s:
    s = s.split("base64,")[1]

This means users can legitimately create ImageBlock instances with the prefix, but the OpenAI conversion breaks this use case.