Skip to content

fix(ci): include default toolsets when enabling extra toolsets in mcpchecker#841

Closed
lyarwood wants to merge 1 commit intocontainers:mainfrom
lyarwood:bug-838
Closed

fix(ci): include default toolsets when enabling extra toolsets in mcpchecker#841
lyarwood wants to merge 1 commit intocontainers:mainfrom
lyarwood:bug-838

Conversation

@lyarwood
Copy link
Copy Markdown
Contributor

@lyarwood lyarwood commented Feb 27, 2026

The TOOLSETS env var was only passing the extra toolsets (kubevirt,
kiali) without including the default toolsets (core, config), causing
the MCP server to run without core tools during evaluations.

Fixes: #838

@lyarwood lyarwood changed the title WIP - improve vm_create and add pause actions to fix KubeVirt evals WIP - kubevirt: improve vm_create and add pause actions to fix evals Feb 27, 2026
…checker

The TOOLSETS env var was only passing the extra toolsets (kubevirt,
kiali) without including the default toolsets (core, config), causing
the MCP server to run without core tools during evaluations.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Lee Yarwood <lyarwood@redhat.com>
@lyarwood lyarwood changed the title WIP - kubevirt: improve vm_create and add pause actions to fix evals fix(ci): include default toolsets when enabling extra toolsets in mcpchecker Mar 5, 2026
@lyarwood lyarwood marked this pull request as ready for review March 5, 2026 14:06
@lyarwood
Copy link
Copy Markdown
Contributor Author

lyarwood commented Mar 5, 2026

/cc @Cali0707
/cc @manusa

Would you mind running the kubevirt evals against this?

@manusa
Copy link
Copy Markdown
Member

manusa commented Mar 5, 2026

/run-mcpchecker kubevirt

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 5, 2026

mcpchecker MCP Evaluation Results

Commit: 09164d0
Summary: 6/12 tasks passed (50.0%)

Metric Result
Tasks Passed 6/12
Assertions Passed 27/33
Overall ❌ Failed

View full results

@ksimon1
Copy link
Copy Markdown
Contributor

ksimon1 commented Mar 26, 2026

/close
replaced by #959

@lyarwood lyarwood closed this Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KubeVirt eval failures

3 participants