feat(inference): add inferenceClass option to inference.LLM

[adrian-cowham]

Description

Adds an optional inferenceClass?: 'priority' | 'standard' option to inference.LLM, letting callers opt into the LiveKit Agent Gateway's priority/standard routing tiers. When set, the outbound request includes an X-LiveKit-Inference-Priority header; when unset, nothing is emitted (behavior unchanged).

The option is available in two places, mirroring the existing precedence pattern for parallelToolCalls/toolChoice in the same file:

1. Constructor — applies to every chat() call made by that LLM instance.
2. Per-call on LLM.chat() — overrides the constructor value for a single request.

Changes Made

• InferenceLLMOptions gets a new optional inferenceClass?: InferenceClass field; InferenceClass is a new exported type alias ('priority' | 'standard').
• LLM constructor accepts inferenceClass and stores it on this.opts without a default.
• LLM.chat() accepts inferenceClass, resolves it as callArg !== undefined ? callArg : this.opts.inferenceClass, and forwards the resolved value to LLMStream.
• LLMStream.run() sets extraHeaders[INFERENCE_PRIORITY_HEADER] = inferenceClass immediately after the existing provider-header block, only when the resolved value is defined.
• Header name literals (X-LiveKit-Inference-Provider, X-LiveKit-Inference-Priority) are consolidated as INFERENCE_PROVIDER_HEADER / INFERENCE_PRIORITY_HEADER constants in agents/src/inference/utils.ts, co-located with the other gateway-concerned helpers.
• type InferenceClass is re-exported from agents/src/inference/index.ts.
• Adds agents/src/inference/llm.test.ts with 7 explicit it(...) tests covering every observable precedence state (no-value, constructor-only × 2, per-call-only × 2, per-call-overrides-constructor × 2), per the project's "explicit tests, no loop-over-cases" convention.

Usage

// Default applied to every chat() call
const llm = new inference.LLM({ model: 'openai/gpt-4o-mini', inferenceClass: 'priority' });
await llm.chat({ chatCtx });                             // header = 'priority'

// Per-call override
await llm.chat({ chatCtx, inferenceClass: 'standard' }); // header = 'standard'

// No default, opt into priority for one call
const llm2 = new inference.LLM({ model: 'openai/gpt-4o-mini' });
await llm2.chat({ chatCtx, inferenceClass: 'priority' }); // header = 'priority'
await llm2.chat({ chatCtx });                             // no header emitted

Pre-Review Checklist

• Build passes: pnpm build, pnpm format:check, pnpm lint (no new warnings) pass locally.
• AI-generated code reviewed: minimal, narrowly-scoped diff; no throwaway comments.
• Changes explained: scoped to a single gateway-routing option.
• Scope appropriate: only the inference.LLM path is touched — inference.STT, inference.TTS, and FallbackAdapter are intentionally out of scope for this PR.

Testing

• Automated tests added: agents/src/inference/llm.test.ts (7/7 pass).
• All inference/voice/LLM unit tests pass. (Unrelated pre-existing plugin-test failures on main — e.g. cerebras/mistral — are not affected by this change.)

[changeset-bot]

⚠️ No Changeset found

Latest commit: 9124309

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 3 additional findings.