Skip to content

Release v1.48.0 - #5079

Merged
VascoSch92 merged 4 commits into
mainfrom
rel-1.48.0
Sep 15, 2026
Merged

Release v1.48.0#5079
VascoSch92 merged 4 commits into
mainfrom
rel-1.48.0

Conversation

@all-hands-bot

@all-hands-bot all-hands-bot commented Sep 15, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Collaborator

Release v1.48.0

This PR prepares the release for version 1.48.0.

Started by: @VascoSch92

Release Checklist

  • Version set to 1.48.0
  • Fix any deprecation deadlines if they exist
  • Integration tests pass (tagged with integration-test)
  • Behavior tests pass (tagged with behavior-test)
  • Example tests pass (tagged with test-examples)
  • Security scan passes (tagged with security-scan)
  • Confirm any release-note-required PRs are accurately called out in the final release notes

What happens on merge

When this PR is merged, the create-release.yml workflow will automatically:

  1. Create a GitHub release with tag v1.48.0 and auto-generated notes, plus an explicit preamble for merged release-note-required PRs
  2. Explicitly dispatch PyPI, npm, GitHub Packages, agent-server image, and binary publishers against tag v1.48.0
  3. Trigger version-bump-prs.yml after successful PyPI publication
🐳 Agent Server images for this PR — GHCR package, pull/run commands, and all pushed tags (click to expand)

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python-slim amd64, arm64 python-node-runtime Link
python amd64, arm64 python-node-runtime Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:7d44742-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-7d44742-python \
  ghcr.io/openhands/agent-server:7d44742-python

All tags pushed for this build

ghcr.io/openhands/agent-server:7d44742-golang-amd64
ghcr.io/openhands/agent-server:7d44742b7853255c6274b71a452cb96ad9331655-golang-amd64
ghcr.io/openhands/agent-server:rel-1.48.0-golang-amd64
ghcr.io/openhands/agent-server:7d44742-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:7d44742-golang-arm64
ghcr.io/openhands/agent-server:7d44742b7853255c6274b71a452cb96ad9331655-golang-arm64
ghcr.io/openhands/agent-server:rel-1.48.0-golang-arm64
ghcr.io/openhands/agent-server:7d44742-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:7d44742-java-amd64
ghcr.io/openhands/agent-server:7d44742b7853255c6274b71a452cb96ad9331655-java-amd64
ghcr.io/openhands/agent-server:rel-1.48.0-java-amd64
ghcr.io/openhands/agent-server:7d44742-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:7d44742-java-arm64
ghcr.io/openhands/agent-server:7d44742b7853255c6274b71a452cb96ad9331655-java-arm64
ghcr.io/openhands/agent-server:rel-1.48.0-java-arm64
ghcr.io/openhands/agent-server:7d44742-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:7d44742-python-amd64
ghcr.io/openhands/agent-server:7d44742b7853255c6274b71a452cb96ad9331655-python-amd64
ghcr.io/openhands/agent-server:rel-1.48.0-python-amd64
ghcr.io/openhands/agent-server:7d44742-python-node-runtime-amd64
ghcr.io/openhands/agent-server:7d44742-python-arm64
ghcr.io/openhands/agent-server:7d44742b7853255c6274b71a452cb96ad9331655-python-arm64
ghcr.io/openhands/agent-server:rel-1.48.0-python-arm64
ghcr.io/openhands/agent-server:7d44742-python-node-runtime-arm64
ghcr.io/openhands/agent-server:7d44742-python-slim-amd64
ghcr.io/openhands/agent-server:7d44742b7853255c6274b71a452cb96ad9331655-python-slim-amd64
ghcr.io/openhands/agent-server:rel-1.48.0-python-slim-amd64
ghcr.io/openhands/agent-server:7d44742-python-node-runtime-slim-amd64
ghcr.io/openhands/agent-server:7d44742-python-slim-arm64
ghcr.io/openhands/agent-server:7d44742b7853255c6274b71a452cb96ad9331655-python-slim-arm64
ghcr.io/openhands/agent-server:rel-1.48.0-python-slim-arm64
ghcr.io/openhands/agent-server:7d44742-python-node-runtime-slim-arm64
ghcr.io/openhands/agent-server:7d44742-golang
ghcr.io/openhands/agent-server:7d44742b7853255c6274b71a452cb96ad9331655-golang
ghcr.io/openhands/agent-server:rel-1.48.0-golang
ghcr.io/openhands/agent-server:7d44742-golang_tag_1.21-bookworm
ghcr.io/openhands/agent-server:7d44742-java
ghcr.io/openhands/agent-server:7d44742b7853255c6274b71a452cb96ad9331655-java
ghcr.io/openhands/agent-server:rel-1.48.0-java
ghcr.io/openhands/agent-server:7d44742-eclipse-temurin_tag_17-jdk
ghcr.io/openhands/agent-server:7d44742-python-slim
ghcr.io/openhands/agent-server:7d44742b7853255c6274b71a452cb96ad9331655-python-slim
ghcr.io/openhands/agent-server:rel-1.48.0-python-slim
ghcr.io/openhands/agent-server:7d44742-python-node-runtime-slim
ghcr.io/openhands/agent-server:7d44742-python
ghcr.io/openhands/agent-server:7d44742b7853255c6274b71a452cb96ad9331655-python
ghcr.io/openhands/agent-server:rel-1.48.0-python
ghcr.io/openhands/agent-server:7d44742-python-node-runtime

About Multi-Architecture Support

  • Each variant tag (e.g., 7d44742-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., 7d44742-python-amd64) are also available if needed

@github-actions @openhands-agent
Release v1.48.0
364a920 
Co-authored-by: openhands <openhands@all-hands.dev>
@all-hands-bot all-hands-bot added integration-test Runs the integration tests and comments the results test-examples Run all applicable "examples/" files. Expensive operation. behavior-test security-scan Run the release security scan (approval-drift + supply-chain) on this PR labels Sep 15, 2026
@github-actions

github-actions Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Hi! I started running the integration tests on your PR. You will receive a comment with the results shortly.

@github-actions

github-actions Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Hi! I started running the behavior tests on your PR. You will receive a comment with the results shortly.

@all-hands-bot

all-hands-bot commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator Author

👋 This PR needs a couple of things fixed before OpenHands can review it:

  • the PR description's HUMAN: section needs at least 20 characters describing what you tested, not just the template placeholder

Push an update once this is addressed and this check re-runs automatically.

This is an automated check - no AI was used to generate this comment.

@github-actions

github-actions Bot commented Sep 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

REST API breakage checks (OpenAPI) — ✅ PASSED

Result: ✅ PASSED

Action log

@github-actions

github-actions Bot commented Sep 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🔒 Release Security Scan

🔒 Approval drift (time-of-check vs time-of-use)

❌ 8 finding(s)

Baseline: v1.47.0 — 49 merged PR(s) in range.

⚠️ 1 first-parent commit(s) in range did not match the (#N) squash convention and were not audited (a merge/rebase merge, or a lost (#N) suffix). These are a blind spot — inspect them:

  • Release v1.48.0
PR finding detail
#4516 changed-after-approval last approval by @neubig was on 5a3978b, but merged head was a569b2960 — commits landed after review
#4724 changed-after-approval last approval by @neubig was on e52669e, but merged head was 860acde55 — commits landed after review
#4931 changed-after-approval last approval by @neubig was on d0df30d, but merged head was a0aa83cd7 — commits landed after review
#4968 changed-after-approval last approval by @enyst was on 8110e26, but merged head was 5bb59ccb3 — commits landed after review
#4927 changed-after-approval last approval by @enyst was on cf1283c, but merged head was 56930cd39 — commits landed after review
#4917 changed-after-approval last approval by @enyst was on 2b97474, but merged head was d934ddcce — commits landed after review
#4947 changed-after-approval last approval by @simonrosenberg was on 9b81079, but merged head was 8fd86e52f — commits landed after review
#4871 changed-after-approval last approval by @simonrosenberg was on 57f8bc9, but merged head was bb578547b — commits landed after review

Audited 49 PR(s): 41 clean, 8 flagged, 0 un-auditable.

📦 Supply-chain dependency diff

✅ no findings

Baseline: v1.47.0
Added: 7, bumped: 9, removed: 5.
(plus 4 internal openhands-* bump(s))

OSV: no known vulns across 20 new/bumped dep(s).

Added dependencies
  • durationpy==0.10
  • httptools==0.8.0
  • itsdangerous==2.2.0
  • k8s-agent-sandbox==0.5.1
  • kubernetes==36.0.0
  • prometheus-client==0.25.0
  • websocket-client==1.9.0
Bumped dependencies
  • cachetools: 5.5.2 → 7.1.8
  • cryptography: 48.0.1 → 50.0.0
  • griffe: 2.0.0 → 2.3.0
  • griffecli: 2.0.0 → 2.3.0
  • griffelib: 2.0.0 → 2.3.0
  • packaging: 25.0 → 26.3
  • pip: 26.1 → 26.2
  • streamlit: 1.54.0 → 1.63.0
  • uvicorn: 0.37.0 → 0.52.4
Internal `openhands-*` bumps
  • openhands-agent-server: 1.47.0 → 1.48.0
  • openhands-sdk: 1.47.0 → 1.48.0
  • openhands-tools: 1.47.0 → 1.48.0
  • openhands-workspace: 1.47.0 → 1.48.0

Deterministic scanners: approval-drift + supply-chain dependency diff. Read-only; no PR code executed.

@github-actions

github-actions Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

🧪 Integration Tests Results

Overall Success Rate: 77.3%
Total Cost: $1.03
Models Tested: 5
Timestamp: 2026-09-15 13:44:55 UTC

📁 Detailed Logs & Artifacts

Click the links below to access detailed agent/LLM logs showing the complete reasoning process for each model. On the GitHub Actions page, scroll down to the 'Artifacts' section to download the logs.

📊 Summary

Model Overall Tests Passed Skipped Total Cost Tokens
litellm_proxy_minimax_MiniMax_M2.7 100.0% 8/8 1 9 $0.00 438,577
litellm_proxy_gemini_3.1_pro_preview 100.0% 9/9 0 9 $0.47 326,158
litellm_proxy_deepseek_deepseek_v4_flash 100.0% 9/9 0 9 $0.03 380,601
litellm_proxy_openai_gpt_5.5 0.0% 0/9 0 9 $0.00 0
litellm_proxy_anthropic_claude_sonnet_4_6 88.9% 8/9 0 9 $0.54 362,877

📋 Detailed Results

litellm_proxy_minimax_MiniMax_M2.7

  • Success Rate: 100.0% (8/8)
  • Total Cost: $0.00
  • Token Usage: prompt: 433,330, completion: 5,247, cache_read: 349,245
  • Run Suffix: litellm_proxy_minimax_MiniMax_M2.7_364a920_minimax_m2_7_run_N9_20260915_134304
  • Skipped Tests: 1

Skipped Tests:

  • t08_image_file_viewing: This test requires a vision-capable LLM model. Please use a model that supports image input.

litellm_proxy_gemini_3.1_pro_preview

  • Success Rate: 100.0% (9/9)
  • Total Cost: $0.47
  • Token Usage: prompt: 321,367, completion: 4,791, cache_read: 130,493, reasoning: 2,946
  • Run Suffix: litellm_proxy_gemini_3.1_pro_preview_364a920_gemini_3_1_pro_run_N9_20260915_134316

litellm_proxy_deepseek_deepseek_v4_flash

  • Success Rate: 100.0% (9/9)
  • Total Cost: $0.03
  • Token Usage: prompt: 375,865, completion: 4,736, cache_read: 252,416, reasoning: 1,164
  • Run Suffix: litellm_proxy_deepseek_deepseek_v4_flash_364a920_deepseek_v4_flash_run_N9_20260915_134318

litellm_proxy_openai_gpt_5.5

  • Success Rate: 0.0% (0/9)
  • Total Cost: $0.00
  • Token Usage: 0
  • Run Suffix: litellm_proxy_openai_gpt_5.5_364a920_gpt_5_5_run_N9_20260915_134322

Failed Tests:

  • t04_git_staging: Test execution failed: Conversation run failed for id=c97e52ff-ea12-4f46-9e8f-72528a720129: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)
  • t03_jupyter_write_file: Test execution failed: Conversation run failed for id=7af56b70-3ffc-4273-85df-bf8d7270e074: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)
  • t09_invoke_skill: Test execution failed: Conversation run failed for id=aa152bd0-934b-42cd-8c88-b19161d690e9: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)
  • t07_interactive_commands: Test execution failed: Conversation run failed for id=1ca30b54-7421-4b1e-bc3f-ac0972703a14: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)
  • t08_image_file_viewing: Test execution failed: Conversation run failed for id=c64af7fe-cac8-4de0-9f35-e4f22c7b3686: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)
  • t06_github_pr_browsing: Test execution failed: Conversation run failed for id=0f90f5c8-092c-4bcf-9b31-e00ea428db89: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)
  • t02_add_bash_hello: Test execution failed: Conversation run failed for id=b16392b1-9315-4837-a981-f0ad0faf1952: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)
  • t05_simple_browsing: Test execution failed: Conversation run failed for id=a2d6d605-5a5a-4972-8ad9-92c386871e99: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)
  • t01_fix_simple_typo: Test execution failed: Conversation run failed for id=f5168717-456e-428d-b979-fddf63a80881: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)

litellm_proxy_anthropic_claude_sonnet_4_6

  • Success Rate: 88.9% (8/9)
  • Total Cost: $0.54
  • Token Usage: prompt: 358,026, completion: 4,851, cache_read: 254,860, cache_write: 103,113, reasoning: 313
  • Run Suffix: litellm_proxy_anthropic_claude_sonnet_4_6_364a920_claude_sonnet_4_6_run_N9_20260915_134314

Failed Tests:

  • t02_add_bash_hello: Shell script is not executable (Cost: $0.06)

@github-actions

github-actions Bot commented Sep 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Coverage

Coverage Report •
FileStmtsMissCoverMissing
openhands-agent-server/openhands/agent_server/docker
   build.py58813078%101, 104–105, 119, 124, 128–130, 134, 139, 156–161, 164, 187, 189, 197, 199–200, 202–206, 208, 212, 215, 218–221, 223–225, 227, 229, 231, 233, 250, 349–350, 354–355, 359–360, 364–365, 378, 380–388, 511, 525, 531–532, 550, 556, 576–577, 579, 653, 667, 670, 674–675, 679–680, 684, 700–702, 705, 711–714, 724, 739, 760, 790, 809–810, 814–816, 870, 981, 991–992, 1173–1175, 1179–1180, 1183, 1200–1205, 1207–1208, 1210–1212, 1215–1216, 1221, 1225, 1271–1273, 1275–1276, 1278–1282, 1295
openhands-sdk/openhands/sdk/conversation/impl
   local_conversation.py10988792%168, 346, 412, 679–680, 722, 764, 1092, 1116–1117, 1122, 1137, 1139, 1243, 1259, 1307, 1333, 1413, 1417–1423, 1488, 1508–1510, 1559, 1578–1580, 1891–1892, 2130, 2133–2134, 2159, 2190, 2196, 2331, 2335, 2338, 2341, 2345–2346, 2350–2351, 2354, 2361, 2381, 2385, 2388, 2407, 2459, 2462, 2494, 2501–2502, 2510, 2514–2516, 2523, 2560–2566, 2569, 2572, 2579, 2680, 2685, 2805–2806, 2824–2825, 2858, 3063, 3067, 3137, 3144–3145
TOTAL44038801782% 

@github-actions

github-actions Bot commented Sep 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🔄 Running Examples with openhands/claude-haiku-4-5-20251001

Generated: 2026-09-15 14:11:36 UTC

Example Status Duration Cost
01_standalone_sdk/02_custom_tools.py ✅ PASS 26.3s $0.03
01_standalone_sdk/03_activate_skill.py ✅ PASS 23.6s $0.03
01_standalone_sdk/05_use_llm_registry.py ✅ PASS 12.5s $0.01
01_standalone_sdk/07_mcp_integration.py ✅ PASS 42.0s $0.03
01_standalone_sdk/09_pause_example.py ✅ PASS 14.9s $0.01
01_standalone_sdk/10_persistence.py ✅ PASS 23.9s $0.02
01_standalone_sdk/11_async.py ✅ PASS 38.7s $0.04
01_standalone_sdk/12_custom_secrets.py ✅ PASS 14.4s $0.01
01_standalone_sdk/13_get_llm_metrics.py ✅ PASS 35.0s $0.03
01_standalone_sdk/14_context_condenser.py ✅ PASS 3m 37s $0.23
01_standalone_sdk/17_image_input.py ✅ PASS 28.3s $0.02
01_standalone_sdk/18_send_message_while_processing.py ✅ PASS 28.5s $0.02
01_standalone_sdk/19_llm_routing.py ✅ PASS 18.3s $0.02
01_standalone_sdk/20_stuck_detector.py ✅ PASS 22.5s $0.03
01_standalone_sdk/21_generate_extraneous_conversation_costs.py ✅ PASS 12.1s $0.00
01_standalone_sdk/22_anthropic_thinking.py ✅ PASS 15.6s $0.01
01_standalone_sdk/23_responses_reasoning.py ❌ FAIL
Exit code 1		 7.1s --
01_standalone_sdk/24_planning_agent_workflow.py ✅ PASS 4m 36s $0.25
01_standalone_sdk/25_agent_delegation.py ✅ PASS 1m 6s $0.06
01_standalone_sdk/26_custom_visualizer.py ✅ PASS 28.6s $0.03
01_standalone_sdk/28_ask_agent_example.py ❌ FAIL
Exit code 1		 9.1s --
01_standalone_sdk/29_llm_streaming.py ✅ PASS 33.8s $0.02
01_standalone_sdk/30_tom_agent.py ✅ PASS 11.3s $0.01
01_standalone_sdk/31_iterative_refinement.py ✅ PASS 4m 40s $0.25
01_standalone_sdk/32_configurable_security_policy.py ✅ PASS 21.0s $0.02
01_standalone_sdk/33_hooks/main.py ✅ PASS 37.2s $0.04
01_standalone_sdk/34_critic_example.py ✅ PASS 2m 35s $0.03
01_standalone_sdk/36_event_json_to_openai_messages.py ✅ PASS 10.6s $0.00
01_standalone_sdk/37_llm_profile_store/main.py ✅ PASS 6.3s $0.00
01_standalone_sdk/38_browser_session_recording.py ✅ PASS 41.0s $0.03
01_standalone_sdk/39_llm_fallback.py ✅ PASS 12.9s $0.01
01_standalone_sdk/40_acp_agent_example.py ✅ PASS 47.4s $0.33
01_standalone_sdk/41_task_tool_set.py ✅ PASS 27.4s $0.03
01_standalone_sdk/42_file_based_subagents.py ✅ PASS 53.5s $0.05
01_standalone_sdk/44_model_switching_in_convo.py ✅ PASS 10.7s $0.01
01_standalone_sdk/45_parallel_tool_execution.py ✅ PASS 3m 45s $0.46
01_standalone_sdk/46_agent_settings.py ✅ PASS 13.7s $0.01
01_standalone_sdk/47_defense_in_depth_security.py ✅ PASS 4.3s $0.00
01_standalone_sdk/48_conversation_fork.py ✅ PASS 28.0s $0.01
01_standalone_sdk/49_switch_llm_tool.py ✅ PASS 10.7s $0.04
01_standalone_sdk/50_async_cancellation.py ✅ PASS 14.6s $0.01
01_standalone_sdk/51_agent_hooks/main.py ✅ PASS 28.3s $0.04
01_standalone_sdk/52_dynamic_workflow.py ✅ PASS 2m 36s $0.09
01_standalone_sdk/53_client_defined_tools.py ✅ PASS 16.4s $0.01
01_standalone_sdk/54_goal_completion_loop.py ✅ PASS 40.5s $0.03
01_standalone_sdk/55_persistent_memory.py ✅ PASS 18.9s $0.02
01_standalone_sdk/56_structured_output.py ✅ PASS 28.7s $0.04
01_standalone_sdk/57_prompt_hooks/main.py ✅ PASS 13.3s $0.00
01_standalone_sdk/58_ask_oracle_tool/main.py ✅ PASS 15.6s $0.01
02_remote_agent_server/01_convo_with_local_agent_server.py ✅ PASS 40.4s $0.02
02_remote_agent_server/02_convo_with_docker_sandboxed_server.py ✅ PASS 1m 34s $0.04
02_remote_agent_server/03_browser_use_with_docker_sandboxed_server.py ✅ PASS 1m 39s $0.08
02_remote_agent_server/04_convo_with_api_sandboxed_server.py ✅ PASS 1m 49s $0.05
02_remote_agent_server/06_custom_tool/main.py ✅ PASS 5m 10s $0.03
02_remote_agent_server/07_convo_with_cloud_workspace.py ✅ PASS 1m 26s $0.04
02_remote_agent_server/08_convo_with_apptainer_sandboxed_server.py ✅ PASS 3m 58s $0.02
02_remote_agent_server/09_acp_agent_with_remote_runtime.py ✅ PASS 1m 8s $0.39
02_remote_agent_server/10_cloud_workspace_share_credentials.py ✅ PASS 1m 18s $0.00
02_remote_agent_server/11_conversation_fork.py ✅ PASS 49.8s $0.00
02_remote_agent_server/12_settings_and_secrets_api.py ✅ PASS 2m 20s $0.01
02_remote_agent_server/13_workspace_get_llm.py ✅ PASS 30.0s $0.01
02_remote_agent_server/14_client_defined_tools.py ✅ PASS 32.8s $0.02
02_remote_agent_server/15_openai_compatible_gateway.py ✅ PASS 24.7s $0.01
02_remote_agent_server/16_deferred_init.py ✅ PASS 15.3s $0.01
02_remote_agent_server/17_convo_with_agent_sandbox_server.py ❌ FAIL
Exit code 1		 6.3s --
04_llm_specific_tools/01_gpt5_apply_patch_preset.py ❌ FAIL
Exit code 1		 13.4s --
04_llm_specific_tools/02_gemini_file_tools.py ✅ PASS 43.1s $0.10
05_skills_and_plugins/01_loading_agentskills/main.py ✅ PASS 18.0s $0.02
05_skills_and_plugins/02_loading_plugins/main.py ✅ PASS 23.2s $0.03
05_skills_and_plugins/04_mixed_marketplace_skills/main.py ✅ PASS 6.7s $0.00

❌ Some tests failed

Total: 70 | Passed: 66 | Failed: 4 | Total Cost: $3.36

Failed examples:

  • examples/01_standalone_sdk/23_responses_reasoning.py: Exit code 1
  • examples/01_standalone_sdk/28_ask_agent_example.py: Exit code 1
  • examples/02_remote_agent_server/17_convo_with_agent_sandbox_server.py: Exit code 1
  • examples/04_llm_specific_tools/01_gpt5_apply_patch_preset.py: Exit code 1

View full workflow run

@github-actions

github-actions Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

🧪 Integration Tests Results

Overall Success Rate: 80.0%
Total Cost: $7.38
Models Tested: 5
Timestamp: 2026-09-15 13:52:50 UTC

📁 Detailed Logs & Artifacts

Click the links below to access detailed agent/LLM logs showing the complete reasoning process for each model. On the GitHub Actions page, scroll down to the 'Artifacts' section to download the logs.

📊 Summary

Model Overall Tests Passed Skipped Total Cost Tokens
litellm_proxy_minimax_MiniMax_M2.7 100.0% 5/5 0 5 $0.12 1,364,313
litellm_proxy_gemini_3.1_pro_preview 100.0% 5/5 0 5 $4.16 6,441,893
litellm_proxy_deepseek_deepseek_v4_flash 100.0% 5/5 0 5 $0.35 4,139,791
litellm_proxy_openai_gpt_5.5 0.0% 0/5 0 5 $0.00 0
litellm_proxy_anthropic_claude_sonnet_4_6 100.0% 5/5 0 5 $2.75 3,875,205

📋 Detailed Results

litellm_proxy_minimax_MiniMax_M2.7

  • Success Rate: 100.0% (5/5)
  • Total Cost: $0.12
  • Token Usage: prompt: 1,340,946, completion: 23,367, cache_read: 1,073,596
  • Run Suffix: litellm_proxy_minimax_MiniMax_M2.7_364a920_minimax_m2_7_run_N5_20260915_134354

litellm_proxy_gemini_3.1_pro_preview

  • Success Rate: 100.0% (5/5)
  • Total Cost: $4.16
  • Token Usage: prompt: 6,404,473, completion: 37,420, cache_read: 4,962,883, reasoning: 15,796
  • Run Suffix: litellm_proxy_gemini_3.1_pro_preview_364a920_gemini_3_1_pro_run_N5_20260915_134318

litellm_proxy_deepseek_deepseek_v4_flash

  • Success Rate: 100.0% (5/5)
  • Total Cost: $0.35
  • Token Usage: prompt: 4,099,016, completion: 40,775, cache_read: 3,608,320, reasoning: 13,052
  • Run Suffix: litellm_proxy_deepseek_deepseek_v4_flash_364a920_deepseek_v4_flash_run_N5_20260915_134307

litellm_proxy_openai_gpt_5.5

  • Success Rate: 0.0% (0/5)
  • Total Cost: $0.00
  • Token Usage: 0
  • Run Suffix: litellm_proxy_openai_gpt_5.5_364a920_gpt_5_5_run_N5_20260915_134337

Failed Tests:

  • b02_no_oververification: Test execution failed: Conversation run failed for id=db16ccd9-e7a2-42c9-aaad-23bc810a64b5: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)
  • b01_no_premature_implementation: Test execution failed: Conversation run failed for id=d6970390-ca1a-4893-bc0b-cf6288c2c507: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)
  • b04_each_tool_call_has_a_concise_explanation: Test execution failed: Conversation run failed for id=44d3c30c-51bf-48d3-80db-e42cf35f5c4d: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)
  • b05_do_not_create_redundant_files: Test execution failed: Conversation run failed for id=e3b1a3b4-8dea-4492-85e9-86ad2a850cba: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)
  • b03_no_useless_backward_compatibility: Test execution failed: Conversation run failed for id=bef968e0-0b5a-4963-bb91-384a69f0dce6: litellm.BadRequestError: {"error":{"message":"litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=openai/gpt-5.5\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: OpenAIException - {\n "error": {\n "message": "You do not have access to the organization tied to the API key.",\n "type": "invalid_request_error",\n "code": "invalid_organization",\n "param": null\n },\n "status": 401\n}No fallback model group found for original model_group=openai/gpt-5.5. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}] LiteLLM Retried: 3 times, LiteLLM Max Retries: 3","type":null,"param":null,"code":"401"}} (Cost: $0.00)

litellm_proxy_anthropic_claude_sonnet_4_6

  • Success Rate: 100.0% (5/5)
  • Total Cost: $2.75
  • Token Usage: prompt: 3,824,754, completion: 50,451, cache_read: 3,486,143, cache_write: 242,179, reasoning: 6,732
  • Run Suffix: litellm_proxy_anthropic_claude_sonnet_4_6_364a920_claude_sonnet_4_6_run_N5_20260915_134308

@enyst

enyst commented Sep 15, 2026

Copy link
Copy Markdown
Member

I went through the security scan and I didn’t spot messed up stuff / supply chain / weird deps or something. I didn’t use my agent, out of principle, but I think maybe next time I can do this scan with a SOTA LLM like Fable 5.1. For better or worse, Anthropic seems to have fairly tough classifiers on it… and human attention, even human skimming, will not scale with diffs as large as these.

@VascoSch92 VascoSch92 mentioned this pull request Sep 15, 2026
Closed
5 tasks
@VascoSch92 VascoSch92 added test-examples Run all applicable "examples/" files. Expensive operation. and removed test-examples Run all applicable "examples/" files. Expensive operation. labels Sep 15, 2026
@github-actions

github-actions Bot commented Sep 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🔄 Running Examples with openhands/claude-haiku-4-5-20251001

Generated: 2026-09-15 20:30:57 UTC

Example Status Duration Cost
01_standalone_sdk/02_custom_tools.py ✅ PASS 22.3s $0.03
01_standalone_sdk/03_activate_skill.py ✅ PASS 23.4s $0.03
01_standalone_sdk/05_use_llm_registry.py ✅ PASS 10.4s $0.01
01_standalone_sdk/07_mcp_integration.py ✅ PASS 34.2s $0.03
01_standalone_sdk/09_pause_example.py ✅ PASS 11.2s $0.01
01_standalone_sdk/10_persistence.py ✅ PASS 25.6s $0.02
01_standalone_sdk/11_async.py ✅ PASS 33.4s $0.04
01_standalone_sdk/12_custom_secrets.py ✅ PASS 13.7s $0.01
01_standalone_sdk/13_get_llm_metrics.py ✅ PASS 31.1s $0.04
01_standalone_sdk/14_context_condenser.py ✅ PASS 2m 43s $0.16
01_standalone_sdk/17_image_input.py ✅ PASS 22.1s $0.02
01_standalone_sdk/18_send_message_while_processing.py ✅ PASS 24.7s $0.02
01_standalone_sdk/19_llm_routing.py ✅ PASS 18.7s $0.02
01_standalone_sdk/20_stuck_detector.py ✅ PASS 16.1s $0.02
01_standalone_sdk/21_generate_extraneous_conversation_costs.py ✅ PASS 11.1s $0.00
01_standalone_sdk/22_anthropic_thinking.py ✅ PASS 16.9s $0.01
01_standalone_sdk/23_responses_reasoning.py ❌ FAIL
Exit code 1		 47.0s --
01_standalone_sdk/24_planning_agent_workflow.py ✅ PASS 4m 20s $0.31
01_standalone_sdk/25_agent_delegation.py ✅ PASS 56.1s $0.05
01_standalone_sdk/26_custom_visualizer.py ✅ PASS 20.5s $0.02
01_standalone_sdk/28_ask_agent_example.py ✅ PASS 32.0s $0.05
01_standalone_sdk/29_llm_streaming.py ✅ PASS 33.2s $0.02
01_standalone_sdk/30_tom_agent.py ✅ PASS 14.9s $0.02
01_standalone_sdk/31_iterative_refinement.py ✅ PASS 4m 38s $0.26
01_standalone_sdk/32_configurable_security_policy.py ✅ PASS 17.0s $0.02
01_standalone_sdk/33_hooks/main.py ✅ PASS 33.8s $0.04
01_standalone_sdk/34_critic_example.py ✅ PASS 1m 57s $0.02
01_standalone_sdk/36_event_json_to_openai_messages.py ✅ PASS 11.9s $0.00
01_standalone_sdk/37_llm_profile_store/main.py ✅ PASS 9.8s $0.00
01_standalone_sdk/38_browser_session_recording.py ✅ PASS 43.4s $0.04
01_standalone_sdk/39_llm_fallback.py ✅ PASS 25.9s $0.01
01_standalone_sdk/40_acp_agent_example.py ✅ PASS 51.0s $0.33
01_standalone_sdk/41_task_tool_set.py ✅ PASS 24.2s $0.03
01_standalone_sdk/42_file_based_subagents.py ✅ PASS 50.9s $0.05
01_standalone_sdk/44_model_switching_in_convo.py ✅ PASS 10.0s $0.01
01_standalone_sdk/45_parallel_tool_execution.py ✅ PASS 5m 14s $0.54
01_standalone_sdk/46_agent_settings.py ✅ PASS 13.4s $0.01
01_standalone_sdk/47_defense_in_depth_security.py ✅ PASS 4.6s $0.00
01_standalone_sdk/48_conversation_fork.py ✅ PASS 22.8s $0.01
01_standalone_sdk/49_switch_llm_tool.py ❌ FAIL
Exit code 1		 3.9s --
01_standalone_sdk/50_async_cancellation.py ✅ PASS 13.9s $0.00
01_standalone_sdk/51_agent_hooks/main.py ✅ PASS 53.2s $0.06
01_standalone_sdk/52_dynamic_workflow.py ✅ PASS 4m 26s $0.16
01_standalone_sdk/53_client_defined_tools.py ✅ PASS 14.8s $0.01
01_standalone_sdk/54_goal_completion_loop.py ✅ PASS 34.5s $0.03
01_standalone_sdk/55_persistent_memory.py ✅ PASS 17.0s $0.02
01_standalone_sdk/56_structured_output.py ✅ PASS 42.3s $0.04
01_standalone_sdk/57_prompt_hooks/main.py ✅ PASS 20.5s $0.00
01_standalone_sdk/58_ask_oracle_tool/main.py ✅ PASS 18.6s $0.01
02_remote_agent_server/01_convo_with_local_agent_server.py ✅ PASS 39.5s $0.02
02_remote_agent_server/02_convo_with_docker_sandboxed_server.py ✅ PASS 1m 34s $0.04
02_remote_agent_server/03_browser_use_with_docker_sandboxed_server.py ✅ PASS 1m 50s $0.13
02_remote_agent_server/04_convo_with_api_sandboxed_server.py ✅ PASS 1m 46s $0.03
02_remote_agent_server/06_custom_tool/main.py ✅ PASS 5m 19s $0.04
02_remote_agent_server/07_convo_with_cloud_workspace.py ✅ PASS 1m 9s $0.03
02_remote_agent_server/08_convo_with_apptainer_sandboxed_server.py ✅ PASS 3m 20s $0.02
02_remote_agent_server/09_acp_agent_with_remote_runtime.py ✅ PASS 1m 43s $0.45
02_remote_agent_server/10_cloud_workspace_share_credentials.py ✅ PASS 1m 6s $0.00
02_remote_agent_server/11_conversation_fork.py ✅ PASS 41.1s $0.00
02_remote_agent_server/12_settings_and_secrets_api.py ✅ PASS 2m 11s $0.01
02_remote_agent_server/13_workspace_get_llm.py ✅ PASS 31.8s $0.01
02_remote_agent_server/14_client_defined_tools.py ✅ PASS 27.7s $0.02
02_remote_agent_server/15_openai_compatible_gateway.py ✅ PASS 22.4s $0.01
02_remote_agent_server/16_deferred_init.py ✅ PASS 22.0s $0.01
04_llm_specific_tools/01_gpt5_apply_patch_preset.py ❌ FAIL
Exit code 1		 8.7s --
04_llm_specific_tools/02_gemini_file_tools.py ✅ PASS 36.4s $0.08
05_skills_and_plugins/01_loading_agentskills/main.py ✅ PASS 16.5s $0.02
05_skills_and_plugins/02_loading_plugins/main.py ✅ PASS 21.7s $0.03
05_skills_and_plugins/04_mixed_marketplace_skills/main.py ✅ PASS 3.8s $0.00

❌ Some tests failed

Total: 69 | Passed: 66 | Failed: 3 | Total Cost: $3.56

Failed examples:

  • examples/01_standalone_sdk/23_responses_reasoning.py: Exit code 1
  • examples/01_standalone_sdk/49_switch_llm_tool.py: Exit code 1
  • examples/04_llm_specific_tools/01_gpt5_apply_patch_preset.py: Exit code 1

View full workflow run

@VascoSch92
VascoSch92 enabled auto-merge (squash) September 15, 2026 20:36
@VascoSch92
VascoSch92 merged commit 481dfb3 into main Sep 15, 2026
89 of 97 checks passed
@VascoSch92
VascoSch92 deleted the rel-1.48.0 branch September 15, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@VascoSch92 VascoSch92 VascoSch92 approved these changes

Assignees

No one assigned

Labels

behavior-test integration-test Runs the integration tests and comments the results security-scan Run the release security scan (approval-drift + supply-chain) on this PR test-examples Run all applicable "examples/" files. Expensive operation.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@all-hands-bot @enyst @VascoSch92 @neubig