feat(llamaindex): Instrumentation adjustment for Otel GenAI semconv support #3979
Merged
max-deygin-traceloop merged 10 commits intomainfrom Apr 16, 2026
Merged
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds pure utilities to normalize LlamaIndex messages/responses into OpenTelemetry GenAI semconv JSON, integrates provider detection, message/token/finish-reason extraction and propagation into instrumentation and event emission, and extends tests to validate the new semconv attributes and behaviors. Changes
Sequence DiagramsequenceDiagram
participant Client
participant Instrumentor as Custom LLM Instrumentor
participant MsgUtils as Message Utilities
participant RespUtils as Response Utilities
participant Span as Span/Attributes
participant Event as EventEmitter
Client->>Instrumentor: _handle_request(instance, args, kwargs)
Instrumentor->>RespUtils: detect_provider_name(instance)
RespUtils-->>Instrumentor: provider_name
Instrumentor->>MsgUtils: build_input_messages(messages)
MsgUtils-->>Instrumentor: messages JSON
Instrumentor->>Span: set_attribute(gen_ai.input.messages, JSON)
Instrumentor->>Span: set_attribute(gen_ai.request.*)
Client->>Instrumentor: _handle_response(response)
Instrumentor->>RespUtils: extract_response_id(response.raw)
RespUtils-->>Instrumentor: response_id
Instrumentor->>RespUtils: extract_token_usage(response.raw)
RespUtils-->>Instrumentor: TokenUsage
Instrumentor->>RespUtils: extract_finish_reasons(response.raw)
RespUtils-->>Instrumentor: finish_reasons[]
Instrumentor->>MsgUtils: build_output_message(response.message, finish_reason)
MsgUtils-->>Instrumentor: output message JSON
Instrumentor->>Span: set_attribute(gen_ai.output.messages, JSON)
Instrumentor->>Event: emit_chat_response_events(event, provider_name)
Event->>RespUtils: extract_finish_reasons(event.response.raw)
RespUtils-->>Event: finish_reasons[]
Event->>Span: emit log with gen_ai.provider.name
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/span_utils.py (1)
68-82:⚠️ Potential issue | 🟡 MinorPotential
AttributeErrorifresponse.messageisNone.Per the context snippet from
_message_utils.py,build_output_message()accessesresponse_message.roleandresponse_message.contentwithout null checking. Ifresponse.messageisNone(which can happen with some LLM responses), this will raise anAttributeError.Consider adding a guard or falling back to
build_completion_output_message():🛡️ Proposed fix
`@dont_throw` def set_llm_chat_response(event, span) -> None: if not span.is_recording(): return response = event.response if should_send_prompts(): fr = None try: finish_reasons = extract_finish_reasons(response.raw) if response.raw else [] fr = finish_reasons[0] if finish_reasons else None except Exception: pass - output_msg = build_output_message(response.message, finish_reason=fr) + if response.message is not None: + output_msg = build_output_message(response.message, finish_reason=fr) + else: + output_msg = build_completion_output_message(getattr(response, "text", "") or "", finish_reason=fr) span.set_attribute(GenAIAttributes.GEN_AI_OUTPUT_MESSAGES, json.dumps([output_msg]))🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/span_utils.py` around lines 68 - 82, In set_llm_chat_response, avoid AttributeError when response.message can be None: check response.message before calling build_output_message and if it's None call build_completion_output_message (or otherwise construct a safe output message from response/response.raw), preserving the finish_reason logic that uses extract_finish_reasons(response.raw); then set the attribute GenAIAttributes.GEN_AI_OUTPUT_MESSAGES with the JSON of that safe output message instead of directly passing response.message to build_output_message.
🧹 Nitpick comments (2)
packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/custom_llm_instrumentor.py (1)
170-174: Minor inconsistency in completion input message construction.For completions, the message is manually constructed with
{"type": "text", "content": text}even whentextcould beNone. This differs frombuild_input_messages()used for chat, which calls_content_to_parts()that returns[]forNonecontent.Consider using a consistent pattern:
♻️ Suggested consistency fix
elif llm_request_type == LLMRequestTypeValues.COMPLETION and args: prompt = args[0] text = prompt[0] if isinstance(prompt, list) else prompt - msg = [{"role": "user", "parts": [{"type": "text", "content": text}]}] + parts = [{"type": "text", "content": text}] if text else [] + msg = [{"role": "user", "parts": parts}] span.set_attribute(GenAIAttributes.GEN_AI_INPUT_MESSAGES, json.dumps(msg))🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/custom_llm_instrumentor.py` around lines 170 - 174, In the COMPLETION branch (LLMRequestTypeValues.COMPLETION) avoid creating a {"type":"text","content": text} entry when text is None; instead reuse the same pattern as build_input_messages()/_content_to_parts() so parts become [] for None. Update the block that builds msg (currently using prompt, text, msg, span.set_attribute) to compute parts = _content_to_parts(text) or explicitly set parts = [] when text is None, then set msg = [{"role":"user","parts": parts}] before json.dumps and span.set_attribute to keep behavior consistent with chat message construction.packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/_message_utils.py (1)
144-150: Minor redundancy in_maybe_wrap_tool_response.The conditional
if parts else ""on line 149 is redundant since line 147-148 already returns early whenpartsis falsy.♻️ Suggested simplification
def _maybe_wrap_tool_response(msg: Any, parts: List[Dict]) -> List[Dict]: """Wrap content as tool_call_response for tool-role messages if tool_call_id present.""" tool_call_id = getattr(msg, "additional_kwargs", {}).get("tool_call_id") if not tool_call_id or not parts: return parts - response_content = parts[0].get("content", "") if parts else "" + response_content = parts[0].get("content", "") return [{"type": "tool_call_response", "id": tool_call_id, "response": response_content}]🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/_message_utils.py` around lines 144 - 150, The function _maybe_wrap_tool_response has a redundant conditional when computing response_content; remove the unnecessary "if parts else ''" fallback since the earlier check "if not tool_call_id or not parts: return parts" guarantees parts is truthy, so compute response_content = parts[0].get("content", "") and return the wrapped dict using tool_call_id and response_content.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In
`@packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/span_utils.py`:
- Around line 68-82: In set_llm_chat_response, avoid AttributeError when
response.message can be None: check response.message before calling
build_output_message and if it's None call build_completion_output_message (or
otherwise construct a safe output message from response/response.raw),
preserving the finish_reason logic that uses
extract_finish_reasons(response.raw); then set the attribute
GenAIAttributes.GEN_AI_OUTPUT_MESSAGES with the JSON of that safe output message
instead of directly passing response.message to build_output_message.
---
Nitpick comments:
In
`@packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/_message_utils.py`:
- Around line 144-150: The function _maybe_wrap_tool_response has a redundant
conditional when computing response_content; remove the unnecessary "if parts
else ''" fallback since the earlier check "if not tool_call_id or not parts:
return parts" guarantees parts is truthy, so compute response_content =
parts[0].get("content", "") and return the wrapped dict using tool_call_id and
response_content.
In
`@packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/custom_llm_instrumentor.py`:
- Around line 170-174: In the COMPLETION branch
(LLMRequestTypeValues.COMPLETION) avoid creating a {"type":"text","content":
text} entry when text is None; instead reuse the same pattern as
build_input_messages()/_content_to_parts() so parts become [] for None. Update
the block that builds msg (currently using prompt, text, msg,
span.set_attribute) to compute parts = _content_to_parts(text) or explicitly set
parts = [] when text is None, then set msg = [{"role":"user","parts": parts}]
before json.dumps and span.set_attribute to keep behavior consistent with chat
message construction.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 186aefe5-3a93-4546-84f6-692d53e2bd6c
⛔ Files ignored due to path filters (1)
packages/opentelemetry-instrumentation-llamaindex/uv.lockis excluded by!**/*.lock
📒 Files selected for processing (15)
packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/_message_utils.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/_response_utils.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/custom_llm_instrumentor.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/dispatcher_wrapper.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/event_emitter.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/event_models.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/span_utils.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_agents.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_custom_llm_semconv.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_event_emitter_semconv.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_message_utils.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_none_content_fix.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_response_utils.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_semconv_migration.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_structured_llm.py
max-deygin-traceloop
force-pushed
the
max/tlp-2032-llamaindex-python-instrumentation
branch
from
7cf200e to
1c36c34
Compare
max-deygin-traceloop
force-pushed
the
max/tlp-2032-llamaindex-python-instrumentation
branch
from
d34ebaf to
bc5e5e2
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/span_utils.py (1)
49-59:⚠️ Potential issue | 🟠 MajorUnwrap
StructuredLLMbefore resolvinggen_ai.provider.name.Provider detection happens on the outer
model_dict, but model/temperature are intentionally read frommodel_dict["llm"]. For wrapped models this can emit the wrapper class as the provider instead of the underlying vendor, so the span ends up internally inconsistent.Suggested fix
model_dict = event.model_dict span.set_attribute(GenAIAttributes.GEN_AI_OPERATION_NAME, "chat") - class_name = model_dict.get("class_name") - provider = detect_provider_name(class_name) + underlying_model_dict = model_dict.get("llm", model_dict) + provider = detect_provider_name( + underlying_model_dict.get("class_name") or model_dict.get("class_name") + ) if provider: span.set_attribute(GenAIAttributes.GEN_AI_PROVIDER_NAME, provider) # For StructuredLLM, the model and temperature are nested under model_dict.llm - if "llm" in model_dict: - model_dict = model_dict.get("llm", {}) + model_dict = underlying_model_dictBased on learnings: Instrumentation packages should leverage the semantic conventions package to generate spans and tracing data compliant with OpenTelemetry semantic conventions.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/span_utils.py` around lines 49 - 59, The provider detection is happening on the outer model_dict which can be a StructuredLLM wrapper; move the unwrap logic up so the code uses the inner model info for provider resolution: first, if "llm" in model_dict then set model_dict = model_dict.get("llm", {}), then obtain class_name and call detect_provider_name(class_name) and set GenAIAttributes.GEN_AI_PROVIDER_NAME on the span (span.set_attribute). Update references to model_dict and class_name accordingly so provider reflects the underlying vendor rather than the wrapper class.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/dispatcher_wrapper.py`:
- Line 91: The provider_name field is only set in the LLMChatStartEvent branch
so rerank (and the other branches around the same pattern) emits None and falls
back to "llamaindex"; update the code to set self.provider_name to the resolved
provider in every branch that emits events (e.g., in the rerank handling code
path and the other event branches referenced) before calling
event_emitter._event_attributes(), so gen_ai.provider.name receives the resolved
provider. Locate uses of provider_name, the LLMChatStartEvent branch, and the
rerank handling code in dispatcher_wrapper.py and assign the same resolved
provider value (the one you derive for LLMChatStartEvent) to self.provider_name
in those branches as well. Ensure the assignment happens prior to calling
event_emitter._event_attributes() so all emitted rerank and other events carry
the correct provider_name.
---
Outside diff comments:
In
`@packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/span_utils.py`:
- Around line 49-59: The provider detection is happening on the outer model_dict
which can be a StructuredLLM wrapper; move the unwrap logic up so the code uses
the inner model info for provider resolution: first, if "llm" in model_dict then
set model_dict = model_dict.get("llm", {}), then obtain class_name and call
detect_provider_name(class_name) and set GenAIAttributes.GEN_AI_PROVIDER_NAME on
the span (span.set_attribute). Update references to model_dict and class_name
accordingly so provider reflects the underlying vendor rather than the wrapper
class.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: d2f99b53-6390-414a-9a22-0c38ad4a2b63
⛔ Files ignored due to path filters (1)
packages/opentelemetry-instrumentation-llamaindex/uv.lockis excluded by!**/*.lock
📒 Files selected for processing (16)
packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/_message_utils.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/_response_utils.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/custom_llm_instrumentor.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/dispatcher_wrapper.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/event_emitter.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/event_models.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/span_utils.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_agents.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_custom_llm_semconv.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_event_emitter_semconv.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_finish_reasons.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_message_utils.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_none_content_fix.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_response_utils.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_semconv_migration.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_structured_llm.py
✅ Files skipped from review due to trivial changes (2)
- packages/opentelemetry-instrumentation-llamaindex/tests/test_structured_llm.py
- packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/event_models.py
🚧 Files skipped from review as they are similar to previous changes (7)
- packages/opentelemetry-instrumentation-llamaindex/tests/test_agents.py
- packages/opentelemetry-instrumentation-llamaindex/tests/test_none_content_fix.py
- packages/opentelemetry-instrumentation-llamaindex/tests/test_event_emitter_semconv.py
- packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/_response_utils.py
- packages/opentelemetry-instrumentation-llamaindex/tests/test_custom_llm_semconv.py
- packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/_message_utils.py
- packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/custom_llm_instrumentor.py
max-deygin-traceloop
changed the title
…ex-python-instrumentation
OzBenSimhonTraceloop
approved these changes
Apr 15, 2026
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/span_utils.py`:
- Around line 132-134: The current branch assumes event.query has .query_str and
drops plain string rerank queries; update the block in span_utils.py (around
should_send_prompts(), GenAIAttributes.GEN_AI_INPUT_MESSAGES, and
emit_rerank_message_event()) to check the type of event.query: if it's a str use
that string as the message content, otherwise read event.query.query_str from
the query object, then build the msg list and json.dumps it before calling
span.set_attribute so string rerank queries are included as
gen_ai.input.messages.
- Around line 49-52: The provider detection is happening before unwrapping
nested StructuredLLM metadata so model_dict.get("class_name") misses cases like
{"llm": {"class_name": "OpenAI", ...}}; modify the logic in span_utils.py to
perform the StructuredLLM unwrapping first (ensure you detect and replace
model_dict with the nested llm dict when class/type indicates StructuredLLM),
then call class_name = model_dict.get("class_name") and provider =
detect_provider_name(class_name) and finally set
span.set_attribute(GenAIAttributes.GEN_AI_PROVIDER_NAME, provider) only after
provider is computed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 7d10d0f4-37c2-43cc-af11-8e3b98411a8b
📒 Files selected for processing (7)
packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/_message_utils.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/custom_llm_instrumentor.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/event_emitter.pypackages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/span_utils.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_custom_llm_semconv.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_message_utils.pypackages/opentelemetry-instrumentation-llamaindex/tests/test_semconv_migration.py
✅ Files skipped from review due to trivial changes (1)
- packages/opentelemetry-instrumentation-llamaindex/tests/test_message_utils.py
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/_message_utils.py
Comment on lines
+49
to
+52
| class_name = model_dict.get("class_name") | ||
| provider = detect_provider_name(class_name) | ||
| if provider: | ||
| span.set_attribute(GenAIAttributes.GEN_AI_PROVIDER_NAME, provider) |
There was a problem hiding this comment.
Move provider detection after unwrapping StructuredLLM.
class_name is read before the nested llm metadata is unwrapped, so {"llm": {"class_name": "OpenAI", ...}} never sets gen_ai.provider.name.
Suggested fix
- class_name = model_dict.get("class_name")
- provider = detect_provider_name(class_name)
- if provider:
- span.set_attribute(GenAIAttributes.GEN_AI_PROVIDER_NAME, provider)
-
# For StructuredLLM, the model and temperature are nested under model_dict.llm
if "llm" in model_dict:
model_dict = model_dict.get("llm", {})
+
+ class_name = model_dict.get("class_name")
+ provider = detect_provider_name(class_name)
+ if provider:
+ span.set_attribute(GenAIAttributes.GEN_AI_PROVIDER_NAME, provider)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/span_utils.py`
around lines 49 - 52, The provider detection is happening before unwrapping
nested StructuredLLM metadata so model_dict.get("class_name") misses cases like
{"llm": {"class_name": "OpenAI", ...}}; modify the logic in span_utils.py to
perform the StructuredLLM unwrapping first (ensure you detect and replace
model_dict with the nested llm dict when class/type indicates StructuredLLM),
then call class_name = model_dict.get("class_name") and provider =
detect_provider_name(class_name) and finally set
span.set_attribute(GenAIAttributes.GEN_AI_PROVIDER_NAME, provider) only after
provider is computed.
Comment on lines
132
to
+134
| if should_send_prompts(): | ||
| span.set_attribute( | ||
| f"{LLMRequestTypeValues.RERANK.value}.query", | ||
| event.query.query_str, | ||
| ) | ||
| msg = [{"role": "user", "parts": [{"type": "text", "content": event.query.query_str}]}] | ||
| span.set_attribute(GenAIAttributes.GEN_AI_INPUT_MESSAGES, json.dumps(msg)) |
There was a problem hiding this comment.
Handle string rerank queries here too.
emit_rerank_message_event() already accepts both str and query objects. This path assumes .query_str, so a plain string will hit @dont_throw, skip the attribute, and silently lose gen_ai.input.messages.
Suggested fix
if should_send_prompts():
- msg = [{"role": "user", "parts": [{"type": "text", "content": event.query.query_str}]}]
+ query = event.query if isinstance(event.query, str) else event.query.query_str
+ msg = [{"role": "user", "parts": [{"type": "text", "content": query}]}]
span.set_attribute(GenAIAttributes.GEN_AI_INPUT_MESSAGES, json.dumps(msg))🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@packages/opentelemetry-instrumentation-llamaindex/opentelemetry/instrumentation/llamaindex/span_utils.py`
around lines 132 - 134, The current branch assumes event.query has .query_str
and drops plain string rerank queries; update the block in span_utils.py (around
should_send_prompts(), GenAIAttributes.GEN_AI_INPUT_MESSAGES, and
emit_rerank_message_event()) to check the type of event.query: if it's a str use
that string as the message content, otherwise read event.query.query_str from
the query object, then build the msg list and json.dumps it before calling
span.set_attribute so string rerank queries are included as
gen_ai.input.messages.
Contributor
OzBenSimhonTraceloop
left a comment
OzBenSimhonTraceloop
left a comment
There was a problem hiding this comment.
@max-deygin-traceloop Just don't forget to update PR title ofc :)
max-deygin-traceloop
changed the title
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrate
opentelemetry-instrumentation-llamaindexfrom legacy indexed attributes (gen_ai.prompt.{i}.*/gen_ai.completion.{i}.*) to OTel GenAI semconv-compliant JSON format (gen_ai.input.messages/gen_ai.output.messages)._message_utils.py— pure functions for building parts-based message JSON and mapping finish reasons across OpenAI,Cohere, and Anthropic
_response_utils.py— response extraction utilities (tokens, model, provider detection, finish reasons) withmulti-provider support
span_utils.pyto emit JSON messages, setgen_ai.operation.name,gen_ai.provider.name,gen_ai.response.id, andgen_ai.response.finish_reasons(as top-levelstring[])custom_llm_instrumentor.py— replaceGEN_AI_SYSTEMwithGEN_AI_PROVIDER_NAME, add chat input/output messagesupport, fix wrong
max_tokens/top_pattribute mappingsevent_emitter.py— replace OpenAI-only finish reason extraction with shared multi-providerextract_finish_reasons()event_models.py— defaultChoiceEvent.finish_reasonfrom"unknown"to""tool_callsfinish reason with upstream Python instrumentation convention (pass-through, not remapped to singular)gen_ai.response.finish_reasonsis never gated byshould_send_prompts()(metadata, not content)Bugs fixed
custom_llm_instrumentor._handle_request:GEN_AI_REQUEST_MAX_TOKENSwas set tocontext_window(model context size) insteadof
num_output;GEN_AI_REQUEST_TOP_Pwas set tonum_outputinstead of an actual top_p valuecustom_llm_instrumentor._handle_response:extract_finish_reasons()called twice per invocation; chat responses lostrole/tool_call info by using completion format
event_emitter.emit_chat_response_events: finish reason extraction only worked for OpenAI dict responses, silently returned""for Cohere/Anthropic_extract_tool_calls:additional_kwargs.get("tool_calls", [])returnsNonewhen key exists withNonevalue — fixed toget("tool_calls") or []extract_finish_reasons: Cohere raw response objects have non-iterable attributes resemblingchoices— addedisinstanceguard
Test plan
uv run --group test python -m pytest tests/ -q)Summary by CodeRabbit
New Features
Refactor
Tests