alibaba · zpzjzj · Sep 4, 2026
@@ -8,6 +8,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Changed
+- Agent execution now performs a side-effect-free binary and version preflight
+  before each case. Host runtimes are never modified; isolated Claude Code,
+  Codex, and Qwen Code runtimes honor concrete `engine.version` selections,
+  and detected CLI versions are recorded as observed report metadata. The
+  preflight does not validate login state or make model requests.
 - Agent adapters now consume one protocol-aware model connection selected from
   provider configuration. OpenAI- and Anthropic-compatible endpoints under the
   same provider no longer pass through the legacy flattened credential view,

@@ -35,6 +35,17 @@ The engine determines the protocol. For example, `provider: dashscope` with a
 static and do not make a model request. The first real agent run is therefore
 the first validation of a delegated local login.
 
+Before each case, skill-up runs a static runtime preflight: the adapter's
+availability check followed by its `--version` command. With `environment.type:
+none`, the host installation is never modified; a configured concrete version
+is only validated against the host CLI. In an isolated runtime, supported npm
+adapters (`claude_code`, `codex`, and `qwen_code`) install the configured
+exact semantic version before the same preflight; tags and ranges are rejected.
+QoderCLI's installer does not support version
+selection, so `engine.version` remains unsupported there, while its detected
+version is still reported. These commands do not inspect login state or make a
+model request.
+
 ## Current resolution order
 
 The runner path resolves values in these stages:
@@ -44,8 +55,8 @@ The runner path resolves values in these stages:
 3. Build one role-aware `ResolvedAgentConfig`. Legacy slash disambiguation is
    performed once at this point instead of mutating and later repairing the
    loaded eval config.
-4. Resolve the provider-scoped `MODEL` value. A provider-scoped model
-   environment variable currently overrides the YAML model.
+4. Resolve a provider-scoped `MODEL` value as a default when no explicit model
+   was selected from YAML or CLI.
 5. Preserve explicit CLI `--model` and `--api-key` precedence.
 6. Apply the selected adapter's declared protocol and capability contract,
    then resolve credentials and endpoints for that `(provider, protocol)`.
@@ -66,9 +77,10 @@ CLI-mutated eval config. `result.json` retains the legacy `engine_name` and
 `model_name` fields while also recording credential-free
 `requested_configuration`, `applied_configuration`, and optional
 `observed_configuration` objects. Per-session results carry the requested and
-applied values, capability warnings, and an observed `model` only when the
-agent explicitly reports it. An absent observation remains unknown; skill-up
-does not probe authentication or spend tokens to infer local CLI state.
+applied values, capability warnings, an observed `model` only when the agent
+explicitly reports it, and the CLI version returned by static runtime
+inspection. An absent observation remains unknown; skill-up does not probe
+authentication or spend tokens to infer local CLI state.
 An applied provider is recorded only when the adapter actively selects that
 provider. If Codex falls back to local provider selection, both the applied
 provider and model are empty and credentials resolved for the rejected provider
@@ -149,12 +161,10 @@ Tagged Actions and historical commands therefore remain valid.
 
 - The legacy flattened credential lookup remains available to older internal
   callers, but adapter construction uses the protocol-aware connection.
-- Provider-scoped `MODEL` currently overrides an explicit YAML model.
-- `engine.version`, `engine.entry`, and `engine.model.params` now produce
-  warnings when ineffective, but their final implementation/removal is deferred
-  to the installation and schema phases.
-- The actual installed CLI version is not yet recorded as observed runtime
-  configuration.
+- `engine.entry` and `engine.model.params` produce warnings when ineffective;
+  their final implementation or removal is deferred to the schema phase.
+- QoderCLI does not support selecting an installer version, so an explicit
+  `engine.version` is warned and ignored for that adapter.
 
 See [Issue #196](https://github.com/alibaba/skill-up/issues/196) for the staged
 cleanup plan.
@@ -88,6 +88,7 @@ skills:
 # ========== 5. Agent Engine ==========
 engine:
   name: claude_code               # claude_code / codex / qodercli (also accepts qoder-cli) / qwen_code (also accepts qwen-code, qwen)
+  version: 2.1.0                  # Optional concrete CLI version; see version lifecycle below
   model:
     provider: anthropic
     name: claude-sonnet-4-6
@@ -136,6 +137,21 @@ remains a valid Skill. The same fields are supported by `judge.skills`.
 
 `cases.parallelism` is the file-level default. To override it for a single run, use `skill-up run --parallelism N` without modifying `eval.yaml`. Allowed range: **1 to 256**.
 
+### Engine version lifecycle
+
+Before each case, skill-up checks that the selected CLI is installed and reads
+its version with a static `--version` command. This does not validate login or
+make a model request. With `environment.type: none`, skill-up never changes the
+host installation; a configured concrete `engine.version` must already match.
+In an isolated runtime, Claude Code, Codex, and Qwen Code install the requested
+version before validation. QoderCLI versions are observed in reports, but its
+installer does not support selecting `engine.version`, so an explicit value is
+ignored with a warning.
+
+For supported adapters, `engine.version` must be one exact semantic version
+(for example, `2.1.0` or `2.1.0-beta.1`); tags and ranges such as `latest` or
+`^2.1.0` are rejected.
+
 ### Engine kwargs (agent-specific switches)
 
 `engine.kwargs` is a free-form string map. Each agent reads only the keys it recognises; unknown keys are ignored. Unrecognised keys (typos like `bypas_sandbox`) emit a DEBUG log line — run with `-v` to surface them. CLI override: `--engine-kwarg key=value` (alias `--ek`), repeatable. Precedence: `--engine-kwarg` > `engine.kwargs` > default.

@@ -88,6 +88,7 @@ skills:
 # ========== 5. Agent Engine ==========
 engine:
   name: claude_code               # claude_code / codex / qodercli（也兼容 qoder-cli）/ qwen_code（也兼容 qwen-code、qwen）
+  version: 2.1.0                  # 可选的 CLI 具体版本；语义见下文
   # kwargs:
   #   edition: cn                 # qodercli 可选：global（默认）/ cn
   model:
@@ -135,6 +136,17 @@ report:
 
 `cases.parallelism` 是配置文件中的默认用例并行数；临时运行时可以用 `skill-up run --parallelism N` 覆盖它，不需要修改 `eval.yaml`。命令行覆盖值必须在 1 到 256 之间。
 
+### Engine 版本生命周期
+
+每个用例执行前，skill-up 会静态检查 CLI 是否安装，并通过 `--version`
+读取版本；该过程不会验证登录状态，也不会发起模型请求。使用
+`environment.type: none` 时，skill-up 不会修改宿主机安装，配置的具体
+`engine.version` 必须与现有 CLI 匹配。在隔离 runtime 中，Claude Code、
+Codex 和 Qwen Code 会先安装指定版本再校验。QoderCLI 的实际版本仍会记录到
+报告，但其安装器不支持选择 `engine.version`，显式配置会被警告并忽略。
+对支持版本选择的 adapter，`engine.version` 必须是完整、精确的语义化版本
+（例如 `2.1.0` 或 `2.1.0-beta.1`）；`latest`、`^2.1.0` 等 tag 或范围不受支持。
+
 ### 采集 workspace 产物（`collect_artifacts`）
 
 `collect_artifacts` 用 glob 声明要从用例 workspace 采集的文件。每次 Agent 运行后——**无论成功、失败还是超时**——命中的文件都会被下载到：

@@ -17,6 +17,11 @@
 #      otherwise the turn behaves like a brand-new conversation.
 set -euo pipefail
 
+if [[ "${1:-}" == "--version" ]]; then
+  echo "qodercli 0.0.0-test"
+  exit 0
+fi
+
 PROMPT=""
 RESUME=""
 while [[ $# -gt 0 ]]; do

@@ -108,6 +108,7 @@ type SessionResult struct {
     RequestedModel  string
     AppliedModel    string
     Model           string // agent-reported observed model; empty means unknown
+    Version         string // CLI version returned by static runtime inspection
     Warnings        []string
     ExitCode       int
     DurationMs     int64
@@ -145,10 +146,17 @@ rejected Codex provider cannot leak its key or endpoint into local fallback.
 | Custom Engine | `custom` | pass through | no; use `engine.custom.http.url` | none; use `engine.custom.kwargs` |
 
 Unsupported kwargs and invalid supported-kwarg values are removed from the
-adapter input after producing an actionable warning. `engine.version`,
-`engine.entry`, and `engine.model.params` are also reported as currently
-ineffective; version lifecycle enforcement is intentionally deferred to the
-installation phase of issue #196.
+adapter input after producing an actionable warning. `engine.entry` and
+`engine.model.params` are also reported as currently ineffective.
+
+Before a case runs, `Preflight` invokes the adapter's availability check and,
+for built-in CLI agents, a static `--version` command. It never checks login
+state or makes a model request. A `none` runtime only validates the existing
+host binary and never installs or upgrades it. Isolated runtimes install a
+configured version first for Claude Code, Codex, and Qwen Code, then validate
+and record the detected version. QoderCLI still reports its detected version,
+but its installer cannot select a requested version, so the capability layer
+warns and omits that constraint.
 
 ### SessionArtifacts
 

@@ -9,6 +9,7 @@ import (
 	"maps"
 	"os"
 	"path/filepath"
+	"regexp"
 	"slices"
 	"strings"
 
@@ -54,7 +55,8 @@ type SessionResult struct {
 	AppliedProvider   string                  `json:"applied_provider,omitempty"`
 	RequestedModel    string                  `json:"requested_model,omitempty"`
 	AppliedModel      string                  `json:"applied_model,omitempty"`
-	Model             string                  `json:"model,omitempty"` // Agent-reported observed model; empty when unavailable.
+	Model             string                  `json:"model,omitempty"`   // Agent-reported observed model; empty when unavailable.
+	Version           string                  `json:"version,omitempty"` // CLI version observed by the static preflight.
 	Warnings          []string                `json:"warnings,omitempty"`
 	SessionID         string                  `json:"session_id,omitempty"`
 	ExitCode          int                     `json:"exit_code"`
@@ -80,6 +82,31 @@ type SessionResumer interface {
 	RunTurn(ctx context.Context, rt Runtime, opts ExecOptions, message transcript.Message, sessionID string) (*SessionResult, error)
 }
 
+// RuntimeObservation contains metadata obtained from side-effect-free runtime
+// inspection. It does not include authentication or login state.
+type RuntimeObservation struct {
+	Version string
+}
+
+// RuntimeInspector is implemented by agents that can inspect runtime metadata
+// after the executable availability check succeeds.
+type RuntimeInspector interface {
+	InspectRuntime(ctx context.Context, rt Runtime) (RuntimeObservation, error)
+}
+
+// Preflight checks executable availability and then collects optional static
+// runtime metadata. It never probes authentication or makes a model request.
+func Preflight(ctx context.Context, rt Runtime, ag Agent) (RuntimeObservation, error) {
+	if err := ag.Check(ctx, rt); err != nil {
+		return RuntimeObservation{}, err
+	}
+	inspector, ok := ag.(RuntimeInspector)
+	if !ok {
+		return RuntimeObservation{}, nil
+	}
+	return inspector.InspectRuntime(ctx, rt)
+}
+
 // SessionArtifacts holds artifacts produced during an agent session.
 type SessionArtifacts struct {
 	WorkspaceDiff  string         `json:"workspace_diff,omitempty"`
@@ -434,6 +461,23 @@ func formatAgentModel(provider, model string) string {
 	return provider + "/" + model
 }
 
+func versionedPackage(packageName, version string) string {
+	version = strings.TrimSpace(version)
+	if version == "" {
+		return packageName
+	}
+	return packageName + "@" + version
+}
+
+func installedVersionGuard(binary, version string) string {
+	expected := normalizedConfiguredVersion(version)
+	if expected == "" {
+		return ""
+	}
+	return "if command -v " + binary + " >/dev/null 2>&1 && " + binary +
+		" --version 2>&1 | grep -Eq " + shellQuote(`(^|[^0-9A-Za-z])v?`+regexp.QuoteMeta(expected)+`([^0-9A-Za-z.+-]|$)`) + "; then exit 0; fi"
+}
+
 // shellQuote quotes a string for safe POSIX shell usage. Agent commands are
 // always composed for and executed by bash (via the Node/nvm bootstrap), so
 // POSIX quoting is correct even when skill-up itself runs on a Windows host.

@@ -559,6 +559,54 @@ func TestDetectAgentWithResolvedConfig_RequiresMaterialization(t *testing.T) {
 	}
 }
 
+func TestDetectAgentRejectsNonConcreteSupportedVersion(t *testing.T) {
+	t.Parallel()
+
+	for _, version := range []string{"latest", "2.1", "1.2.3.4"} {
+		_, err := DetectAgent("codex", Config{Version: version})
+		if err == nil || !strings.Contains(err.Error(), "exact semantic version") {
+			t.Fatalf("DetectAgent version %q error = %v, want exact semantic version error", version, err)
+		}
+	}
+}
+
+func TestInstalledVersionGuardMatchesCompleteVersion(t *testing.T) {
+	t.Parallel()
+
+	guard := installedVersionGuard("example", "1.2.3")
+	if guard == "" {
+		t.Fatal("installedVersionGuard returned an empty guard")
+	}
+	for _, test := range []struct {
+		name    string
+		version string
+		want    bool
+	}{
+		{name: "exact", version: "1.2.3", want: true},
+		{name: "v prefix", version: "v1.2.3", want: true},
+		{name: "prerelease", version: "1.2.3-beta.1", want: false},
+		{name: "build metadata", version: "1.2.3+build.1", want: false},
+		{name: "four part", version: "1.2.3.4", want: false},
+	} {
+		t.Run(test.name, func(t *testing.T) {
+			t.Parallel()
+			cmd := "example() { printf '%s\\n' " + shellQuote(test.version) + "; }; " + guard + "; exit 1"
+			rt := &runtime.NoneRuntime{}
+			if err := rt.Create(context.Background()); err != nil {
+				t.Fatal(err)
+			}
+			defer func() { _ = rt.Close() }()
+			result, err := rt.Exec(context.Background(), cmd, ExecOptions{})
+			if err != nil {
+				t.Fatalf("execute guard: %v", err)
+			}
+			if got := result.ExitCode == 0; got != test.want {
+				t.Fatalf("guard match for %q = %v, want %v", test.version, got, test.want)
+			}
+		})
+	}
+}
+
 func TestDetectAgentWithResolvedConfig_ConsumesAppliedConnection(t *testing.T) {
 	t.Parallel()
 

@@ -41,7 +41,8 @@ const (
 )
 
 // Capabilities declares the configuration surface consumed by an adapter.
-// Installation/version behavior remains intentionally deferred to issue #196 PR 5.
+// Version support describes adapter selection and enforcement; static version
+// observation is handled by the runtime preflight.
 type Capabilities struct {
 	Protocol        Protocol
 	ModelPolicy     ModelPolicy
@@ -66,12 +67,13 @@ var (
 func CapabilitiesForEngine(engineName string) Capabilities {
 	switch engineName {
 	case agentkind.ClaudeCode, agentkind.ClaudeCodeAlias:
-		return Capabilities{Protocol: ProtocolAnthropic, ModelPolicy: ModelPolicyPassthrough, SupportsBaseURL: true}
+		return Capabilities{Protocol: ProtocolAnthropic, ModelPolicy: ModelPolicyPassthrough, SupportsBaseURL: true, SupportsVersion: agentkind.SupportsVersion(engineName)}
 	case agentkind.Codex:
 		return Capabilities{
 			Protocol:        ProtocolOpenAI,
 			ModelPolicy:     ModelPolicyCodexProvider,
 			SupportsBaseURL: true,
+			SupportsVersion: agentkind.SupportsVersion(engineName),
 			SupportedKwargs: slices.Clone(codexKwargs),
 		}
 	case agentkind.QoderCLI, agentkind.QoderAlias, agentkind.QoderCLIAlias:
@@ -81,7 +83,7 @@ func CapabilitiesForEngine(engineName string) Capabilities {
 			SupportedKwargs: slices.Clone(qoderKwargs),
 		}
 	case agentkind.QwenCode, agentkind.QwenCodeAlias, agentkind.QwenAlias:
-		return Capabilities{Protocol: ProtocolOpenAI, ModelPolicy: ModelPolicyPassthrough, SupportsBaseURL: true}
+		return Capabilities{Protocol: ProtocolOpenAI, ModelPolicy: ModelPolicyPassthrough, SupportsBaseURL: true, SupportsVersion: agentkind.SupportsVersion(engineName)}
 	default:
 		return Capabilities{
 			Protocol:    ProtocolCustom,
@@ -273,7 +275,7 @@ func validateBaseURL(params *credential.ResolvedAgentConfig, capabilities Capabi
 func validateDeferredFields(params *credential.ResolvedAgentConfig, capabilities Capabilities) {
 	if params.Version != "" && !capabilities.SupportsVersion {
 		params.Warnings = appendUniqueWarning(params.Warnings, fmt.Sprintf(
-			"engine %q does not yet enforce engine.version; the configured version %q is not guaranteed",
+			"engine %q does not support engine.version; the configured version %q is ignored",
 			params.Engine, params.Version,
 		))
 	}