Always display the LLM profile in use in PR-review and Slack automation output

[smolpaws]

🐾 Hi, I'm smolpaws — a small cat agent built on OpenHands, filing this on Engel's behalf.

Problem or Use Case

Automations that produce judgment — especially the PR-review automation and the Slack automations — are heavily LLM-dependent, and their output quality varies with the model/profile that ran them. Today, when one of these automations posts a review or a Slack message, there is no indication of which LLM profile produced it. A human reading the result cannot tell whether it came from a strong reviewer model or a weak one, which makes the output hard to calibrate and trust.

This came out of the OpenHands "software factory" discussion: as auto-review and Slack automations become load-bearing, the reader needs to know the provenance of each automated judgment, starting with the model behind it.

Desired Behavior

Every automation that posts a result should always display the LLM profile in use, at minimum for the PR-review and Slack automations:

• The PR-review automation includes the LLM profile (and model) it used, visibly, in the review it posts.
• The Slack automations include the LLM profile (and model) they used in the message they post (or a compact, non-noisy equivalent — e.g. a footer or a hover/detail).
• The label is unambiguous: the reader can map "this review/message" → "this profile/model" without digging into run logs.

Acceptance Criteria

• PR-review automation output shows the LLM profile/model that generated the review.
• Slack automation output shows the LLM profile/model that generated the message.
• The display is consistent across these automations (same field/wording), so it reads the same everywhere.
• It's concise enough not to add noise to the primary content.

Additional Context

• Part of the broader "software factory" direction (Graham Neubig's #proj-automation thread): making automated judgments auditable. This issue is the first, smallest piece — surfacing the model. A follow-on (not this issue) is surfacing the context the automation had, e.g. links to the skills at the correct commit hash.
• Maintainers: if profile/model provenance is better surfaced at the OpenHands automation-service layer vs. inside each automation's prompt/output, please redirect — I filed here because the PR-review and Slack automations live in this repo.

Issue formatting updated by an AI agent (OpenHands) on behalf of @enyst.

[github-actions]

✅ ready-for-dev label applied

This issue meets the readiness criteria and is ready for contribution.
A contributor can now open a PR that links this issue (Fixes #<number>).

This is an automated check — no AI was used to generate this comment.

[enyst]

@OpenHands please see the comment and understand the issue; fix the format if we have enough info to fix it correctly. then take a deep breath and implement the issue red/green. open pr

[openhands-ai]

I'm on it! enyst can track my progress at all-hands.dev

[openhands-ai]

Implemented the fix in the owning repository, OpenHands/extensions.

• PR: fix: show LLM provenance in automation output extensions#547
• Branch: openhands/show-llm-provenance
• Commit: 7a0dba5
• PR is draft, awaiting human review/readiness.

Changes:

• PR-review and Slack monitor outputs now use the same concise footer: LLM profile: <profile> · Model: <model>.
• Selected AUTOMATION_MODEL profiles are resolved and used for spawned conversations.
• Profile/model metadata is persisted for completion, error, fallback, and follow-up output.
• Reviewer bundle version bumped to 1.0.1; generated catalogs updated.
• Issue Always display the LLM profile in use in PR-review and Slack automation output #430 headings were corrected to the required format; it now has enhancement and ready-for-dev labels.

Verification: 802 passed, 24 skipped; sync, compilation, and diff checks passed.