[misc] chore: add optional vLLM cache config for fully-async NPU script

[lxb007981]

What does this PR do?

Disable vLLM's multimodal processor cache by default for vLLM older than 0.22.0 to avoid cache desync across pause/resume. Keep explicit user overrides intact. Otherwise an AssertionError: Expected a cached item for mm_hash='...' will be raised during verl update_weights() calling vllm pause_generation(clear_cache=True). See vllm-project/vllm#43001 for details.

Checklist Before Starting

• Search for similar PRs. Paste at least one query link here: ...

• Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)

  • {modules} include fsdp, megatron, veomni, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data, cfg, reward, fully_async, one_step_off
  • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
  • {type} is in feat, fix, refactor, chore, test
  • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
  • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc.

API and Usage Example

Demonstrate how the API changes if any, and provide usage example(s) if possible.

# Add code snippet or script demonstrating how to use this

Design & Code Changes

Demonstrate the high-level design if this PR is complex, and list the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

• Read the Contribute Guide.
• Apply pre-commit checks: pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always
• Add / Update the documentation.
• Add unit or end-to-end test(s) to the CI workflow to cover all the code. If not feasible, explain why: ...
• Once your PR is ready for CI, send a message in the ci-request channel in the verl Slack workspace. (If not accessible, please try the Feishu group (飞书群).)
• If your PR is related to the recipe submodule, please also update the reference to the submodule commit via git submodule update --remote or cd recipe && git pull origin main.

[gemini-code-assist]

Code Review

This pull request updates configuration parameters for Qwen3-VL-8B training, including increasing the maximum response length to 8192, adjusting the PPO maximum token length per GPU to 10240, and introducing a VLLM_CACHE_CONF configuration option. Feedback on the changes highlights critical bash syntax issues in geo3k_qwen3vl_8b_fsdp2_16_16_npu.sh: a trailing backslash-quote in the commented-out VLLM_CACHE_CONF definition that would cause an unexpected EOF error if uncommented, and a missing line continuation backslash after ${VLLM_CACHE_CONF} within the ACTOR_CONFIG multi-line string.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The trailing backslash-quote (") inside the double-quoted string escapes the closing double quote. If this line is uncommented, the string will remain open, leading to a bash syntax error (unexpected EOF while looking for matching '"'). To avoid this and keep the configuration clean, we should define VLLM_CACHE_CONF without any trailing backslash, and instead handle the line continuation backslash directly within the ACTOR_CONFIG definition. Additionally, ensure we use + instead of ++ as the prefix for overriding configuration values in Hydra.

Suggested change

	# VLLM_CACHE_CONF="+actor_rollout_ref.rollout.engine_kwargs.vllm.mm_processor_cache_gb=0 \"
	# VLLM_CACHE_CONF="+actor_rollout_ref.rollout.engine_kwargs.vllm.mm_processor_cache_gb=0"

References

1. Use + instead of ++ as the prefix for overriding configuration values in Hydra.

[gemini-code-assist]

Add a trailing backslash to ensure proper line continuation within the ACTOR_CONFIG multi-line string, regardless of whether VLLM_CACHE_CONF is empty or populated. This keeps the variable definition clean and prevents syntax/formatting issues.

Suggested change

	${VLLM_CACHE_CONF}
	${VLLM_CACHE_CONF} \

[tardis-key]

The script examples/grpo_trainer/run_qwen3_vl_8b_fsdp.sh is generic. Are there any precision and performance experiments comparing GPU and NPU, before and after the changes?

[tardis-key]

What is the motivation behind these changes? As an example, increasing the number of nodes from 8 to 16 would raise the cost of the experiment.

[lxb007981]

The script examples/grpo_trainer/run_qwen3_vl_8b_fsdp.sh is generic. Are there any precision and performance experiments comparing GPU and NPU, before and after the changes?

Understood. Since we haven't done extensive performance experiments yet, I will keep this PR restricted strictly to the bugfix.

[lxb007981]

The script examples/grpo_trainer/run_qwen3_vl_8b_fsdp.sh is generic. Are there any precision and performance experiments comparing GPU and NPU, before and after the changes?

Hi, I've pushed the updated changes. Could you help take a look? Thanks!

[lxb007981]

The failed ci is fixed by #6937

[tardis-key]

The script examples/grpo_trainer/run_qwen3_vl_8b_fsdp.sh is generic. Are there any precision and performance experiments comparing GPU and NPU, before and after the changes?

Hi, I've pushed the updated changes. Could you help take a look? Thanks!

Now the change only affects NPU. Can you confirm this issue only exists on NPU?
And I think the version detection and parameter override should be done in the Python configuration layer rather than inlined in the shell script.

[lxb007981]

The script examples/grpo_trainer/run_qwen3_vl_8b_fsdp.sh is generic. Are there any precision and performance experiments comparing GPU and NPU, before and after the changes?

Hi, I've pushed the updated changes. Could you help take a look? Thanks!

Now the change only affects NPU. Can you confirm this issue only exists on NPU? And I think the version detection and parameter override should be done in the Python configuration layer rather than inlined in the shell script.

Thanks for the feedback. The upstream vLLM issue vllm-project/vllm#42995 was reproduced on GPU. I moved the parameter override to the python configuration side so that both GPU/NPU could benefit. Would you like to take a look?