Type LLM streaming, message, and tokenizer boundaries

[neubig]

Parent tracking issue: #4904

Desired Behavior

LLM streaming wrappers, provider messages, tokenizer outputs, optional transformer loading, and response events should be represented by explicit protocols or adapter functions. Core LLM logic should consume those typed contracts with direct access instead of probing provider objects dynamically.

The fix in #4772 remains authoritative for the yielded-versus-wrapper completed-response bug; this work should build on or avoid conflicting with it.

Scope

• Streaming/event and tokenizer access in llm.py
• Provider message normalization in message.py and mixins/non_native_fc.py
• OpenAI auth capability access
• Excludes telemetry normalization and retry metadata

Acceptance Criteria

• Streaming completion wrappers and response events have explicit typed contracts.
• Tokenizer and optional-transformer capabilities are isolated behind typed adapters.
• Provider message reasoning/thinking fields are normalized once before core message conversion.
• Tests cover sync/async streams, missing optional capabilities, and supported provider message shapes.
• Changes do not regress or duplicate fix(sdk): keep the completion a Responses stream yielded #4772 and the baseline shrinks for removed calls.

This issue was created by an AI agent (OpenHands) on behalf of the user.

[github-actions]

✅ ready-for-dev label applied

This issue meets the readiness criteria and is ready for contribution.
A contributor can now open a PR that links this issue (Fixes #<number>).

This is an automated check — no AI was used to generate this comment.