Skip to content

[ray-update-2.56.0] Update llm_batch_inference_vision to Ray 2.56.0#887

Merged
Aydin-ab merged 3 commits into
mainfrom
cursor/template-ray-version-3ea0
Jul 8, 2026
Merged

[ray-update-2.56.0] Update llm_batch_inference_vision to Ray 2.56.0#887
Aydin-ab merged 3 commits into
mainfrom
cursor/template-ray-version-3ea0

Conversation

@svc-template-updater

@svc-template-updater svc-template-updater commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Bump llm_batch_inference_vision to Ray 2.56.0.

Changes

  • BUILD.yaml: cluster_env.image_urianyscale/ray-llm:2.56.0-py312-cu130 (was 2.55.1-py311-cu128). Ray 2.56.0 anyscale/ray-llm only ships py312-cu130, so this also moves the Python/CUDA pair (tag verified on Docker Hub).
  • templates/llm_batch_inference_vision/job.yaml: image_urianyscale/ray-llm:2.56.0-py312-cu130.
  • templates/llm_batch_inference_vision/README.md + README.ipynb: in-doc image_uri example → anyscale/ray-llm:2.56.0-py312-cu130.
  • dependencies/template.depsets.yaml: repoint this template's expand entry build_arg_sets ray2551_py311_cu128ray2560_py312_cu130; PyTorch --indexcu130. The ray2560_py312_cu130 bundle + rayllm_2.56.0_py312_cu130.lock base already existed from build(deps): Ray 2.56.0 image base locks (base-only) #833's follow-up, so only this expand pointer + index flipped.
  • templates/llm_batch_inference_vision/requirements.txt: refresh the comment to reference anyscale/ray-llm:2.56.0 and pin huggingface-hub==1.13.0 to match the new base image (was 0.36.2). numpy==1.26.4, pandas==2.3.3, pyarrow==19.0.1, and datasets==3.6.0 kept unchanged — the 2.56.0 ray-llm image still ships numpy 1.x + pyarrow 19 + scipy compiled against numpy 1.x (verified from the CI package diff), so keeping the numpy-1 stack is what actually lets the layered lock reinstall as a no-op.
  • templates/llm_batch_inference_vision/batch_inference_vision.py + batch_inference_vision_scaled.py + README.ipynb/README.md: migrate ray.data.llm.build_llm_processorbuild_processor. Ray 2.56.0 removed the deprecated build_llm_processor alias in [data][llm] Remove guided_decoding, truncate_prompt_tokens, build_llm_processor ray-project/ray#63569.
  • Regenerated templates/llm_batch_inference_vision/python_depset.lock via ./update_deps.sh --name llm_batch_inference_vision_depset_2.56.0_3.12_cu130.

Fix iterations

Two failure→fix cycles (both agent-fixable, not infra):

  1. Build fix the diagrams on stable diffusion notebook #490ImportError: A module that was compiled using NumPy 1.x cannot be run in NumPy 2.2.6 from scipy.io._sparsetools on first import datasets. The Ray 2.56.0 rayllm deplock declares numpy==2.2.6, but the actual anyscale/ray-llm:2.56.0-py312-cu130 image still ships numpy 1.26.4 + pyarrow 19.0.1 (with scipy compiled against numpy 1.x). Fix: pin the layered lock to the runtime versions (numpy==1.26.4, pyarrow==19.0.1), matching the classic "un-pinned floats" gotcha in references/dependencies.md.
  2. Build [template][deps] Adding lock file for ai-libs-training #494ImportError: cannot import name 'build_llm_processor' from 'ray.data.llm'. Ray 2.56.0 removed the deprecated alias ([data][llm] Remove guided_decoding, truncate_prompt_tokens, build_llm_processor ray-project/ray#63569). Fix: rename to build_processor in the two .py files and in the notebook + regenerated README.

Tests / validation

Publish

BUILDKITE_API_TOKEN provided to this Cloud Agent lacks the write_builds scope, so this run cannot programmatically create_build on tmpl-publish (see .claude/skills/template/references/publish-to-backend.md). After merge, a reviewer should trigger a fresh tmpl-publish build (org_slug=anyscale, pipeline_slug=tmpl-publish, branch=master, message=llm_batch_inference_vision) and drive the manual gates dev → staging → production per that reference. Pipeline: https://buildkite.com/anyscale/tmpl-publish.

Co-authored-by: Aydin Abiar <Aydin-ab@users.noreply.github.com>
@svc-template-updater svc-template-updater added ray-update updating template to a new ray version cursor-cloud Generated by Cursor Cloud agents labels Jul 8, 2026
@svc-template-updater

Copy link
Copy Markdown
Contributor Author

/test-template llm_batch_inference_vision

The 2.56.0 anyscale/ray-llm image ships numpy 1.26.4 + pyarrow 19.0.1
(scipy is compiled against numpy 1.x). Upgrading numpy to 2.2.6 via
the layered lock broke scipy on import — pin the runtime versions so
the reinstall is a no-op and keep the numpy-1 ABI intact.

Co-authored-by: Aydin Abiar <Aydin-ab@users.noreply.github.com>
@svc-template-updater

Copy link
Copy Markdown
Contributor Author

/test-template llm_batch_inference_vision

Ray 2.56.0 removed the deprecated build_llm_processor alias in
ray-project/ray#63569; use the canonical build_processor. Updated:
- batch_inference_vision.py / batch_inference_vision_scaled.py
- README.ipynb + regenerated README.md

Co-authored-by: Aydin Abiar <Aydin-ab@users.noreply.github.com>
@svc-template-updater

Copy link
Copy Markdown
Contributor Author

/test-template llm_batch_inference_vision

@svc-template-updater svc-template-updater marked this pull request as ready for review July 8, 2026 01:10
@svc-template-updater svc-template-updater requested a review from a team as a code owner July 8, 2026 01:10
@Aydin-ab Aydin-ab merged commit 55dc66b into main Jul 8, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cursor-cloud Generated by Cursor Cloud agents ray-update updating template to a new ray version

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants