feat(templates): add vlm distillation catalog enrichment template + test#702
Open
gc-anyscale wants to merge 9 commits into
Open
feat(templates): add vlm distillation catalog enrichment template + test#702gc-anyscale wants to merge 9 commits into
gc-anyscale wants to merge 9 commits into
Conversation
Copies the three VLM distillation demo notebooks, their entry-point scripts, and the shared src/ modules from sa-demos/.../vlm-batch-embeddings into a new template directory. This commit is a verbatim transfer — no cleanup, no walmart-content scrub, no renaming, no template-voice rewrites. Subsequent commits will strip internal references, restructure to template conventions, swap the Stage 1 teacher from 32B to 7B, write the orchestrator README, and wire BUILD.yaml + configs + tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- /mnt/shared_storage/walmart-notebooks → /mnt/cluster_storage/vlm-distillation-catalog-enrichment - Walmart / Walmart-class prose → ecommerce / production-scale - Hardcoded LoRA S3 path (anyscale-production-data-cld-...) → None, falling back to the base 3B model unless QWEN_LORA_ADAPTER_DIR is set - console.anyscale.com/cld_... job URLs stripped - User-Agent "anyscale-demo/1.0" → "vlm-distillation-catalog-enrichment/1.0" - cld_/org_/prj_/prodjob_ identifiers scrubbed from notebooks + scripts + src 10 files touched, zero residual references confirmed by grep. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stage 1 is now Qwen2.5-VL-7B with TP=1 + CONCURRENCY=4 (one replica per L4 on a g6.12xlarge), replacing the prior 32B TP=4 + CONCURRENCY=1 layout. The 7B fits on a single L4 in bf16 (~14GB / 24GB), so replica-parallelism beats tensor-parallelism: every GPU runs an independent model every step with no all-reduce overhead. Knock-on changes: - Stage 1 output renamed: vlm_enriched_32b_*.parquet → teacher_7b_enriched_*.parquet - Stage 2 TEACHER_PARQUET path updated to match - Prose throughout (Stage 1 + Stage 2) refers to "7B teacher" / "teacher quality" - Field-flavored optimization narrative (32B baseline metrics, "v2 changes", cost numbers) removed from Stage 1's top docstring — Group B cleanup - Generic sizing reference (`Qwen2.5-VL ships in 3B / 7B / 32B / 72B`) kept Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- templates/vlm-distillation-catalog-enrichment/README.ipynb: top-level
distillation story. 17 cells, runs all three stages end-to-end with sample
output inspection between each. Smoke knobs (N_ROWS, TEACHER_N_ROWS,
CATEGORY) read from env vars so tests.sh can drop to N=20.
- configs/vlm-distillation-catalog-enrichment/{aws,gce}.yaml: CPU head +
g6.12xlarge / g2-standard-48 (4× L4) worker pool, max_workers=2.
- tests/vlm-distillation-catalog-enrichment/tests.sh: nb2py-converts the
README and runs it under the N=20 smoke configuration.
- BUILD.yaml: new entry pinning anyscale/ray-llm:2.55.1-py311-cu128
(matches entity-recognition-with-llms — needs vLLM + Ray Data LLM +
Ray Train), 3600s test timeout.
Also makes Stage 1 + Stage 3 honor N_ROWS / TEACHER_N_ROWS / CATEGORY env
overrides so the same script + job config submits both the smoke and the
full run without code edits (Stage 2 already supported this).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stage notebooks (01/02/03):
- Renamed Stage 2 title from "Smoke Test" framing to template voice
("Stage 2 — Distill the student with FSDP + LoRA"); added orchestrator
README link in each stage's intro
- Added an H1 title cell to Stage 1 (notebook previously started with a
code cell)
- Removed the dead `outs = [...]` literal cell from Stage 3 (the migration
checklist's headline removal)
- Removed the stray markdown-fence-only cell in Stage 1 (duplicated the
prior code cell's output)
- Removed the trailing empty cell in Stage 1
- Updated all stale script-name imports to match the renamed files
(run_enc_vlm_batch_emb_enrich_3b → run_enrich_and_embed,
run_vlm_ft_enrich_3b → run_distill_student_lora,
run_finetune_vlm_enrichment_3b → run_distill_student_lora,
run_vlm_batch_enrich_32b → run_teacher_batch_label)
- Replaced the hardcoded runtime_env working_dir
("/home/ray/default_/sa-demos/.../vlm-batch-embeddings") with "."
- Cleared all baked-in cell outputs from the prior demo runs (which carried
stale /home/ray/default_/sa-demos paths, session URLs, and cluster IDs
embedded in Ray log lines). Outputs will be regenerated on first run.
Scripts:
- Same script-name updates in run_enrich_and_embed.py + run_distill_student_lora.py
docstrings
Assets:
- Removed unreferenced images/32b_baseline_*.png (six stale Ray dashboard
screenshots from the prior 32B run, not used by any markdown in the new
template). The orchestrator README uses an ASCII architecture diagram.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reviewed entity-recognition-with-llms, audio-dataset-curation-llm-judge,
e2e-rag-deepdive, and ecommerce_batch_embeddings, then rewrote our prose
to match the dominant gallery voice: HTML badges (not shields.io markdown),
"⏱️ Time to complete" callout, third-person descriptive narration ("Ray Data
reads...", "`ray.data.llm` wraps the vLLM engine..."), sentence-case section
headers, short paragraphs before each code cell.
Orchestrator README.ipynb:
- Swap shields.io badges for the canonical run-on-anyscale.svg + GitHub
shield HTML used by entity-recognition-with-llms
- "**⏱️ Time to complete**: 45 min" callout
- Intro paragraph + numbered overview of the 3 stages
- "Ray is particularly powerful for this workload because it:" bullet list
matching the audio-dataset-curation README pattern
- Sentence-case section headers (## Architecture, ## Set up, ## Stage 1...,
## Run as a scheduled Anyscale Job, ## Clean up)
- Reworked each Stage section's narration to describe the system in
third-person ("Qwen2.5-VL-7B reads a product image...", "Ray Train + FSDP
fine-tunes Qwen2.5-VL-3B...", "The third stage chains together...")
- Fixed nested f-string quotes that would fail on py3.11 (the pinned image)
Stage 1 notebook:
- Demoted inner H1 headers ("# Preprocess Dataset from Source",
"# vLLM preprocessing") to sentence-case H2 with proper gallery-voice
narration paragraphs
- Inserted a "### Sample row" narration before the example-display cell
- Updated the vLLM config cell from Qwen2.5-VL-3B + concurrency=1 (leftover
from when this notebook was the 3B preview of the 32B teacher) to
Qwen2.5-VL-7B + concurrency=4 so the notebook matches its title and the
Stage 1 script
- Dropped two dead cells: the inline `o = {...}` literal sample and its
paired PIL image display (already flagged in the migration checklist)
- Dropped a duplicate `print(json.dumps(output, indent=2))` orphan cell
- Cleaned typos and stale comments in the vLLM config
Stage 2 notebook:
- Renamed "## Smoke-test config" → "## Configure the run"
- "## Build SFT cache (small subset)" → "## Build the SFT cache"
- "## Sanity check — one row" → "## Sanity check one row"
- "## Run training (smoke scale)" → "## Run training"
- "## Next steps" → "## Submit the full run as an Anyscale Job"
- Dropped self-referential "tests the same code paths as the production
script" repetitions from body cells (kept the framing once in the intro)
- Made the intro paragraph third-person descriptive
Stage 3 notebook:
- Tightened the intro paragraph and dropped the "Smoke notebook for ..."
framing
- Inserted four narration markdown cells between previously-bare code
cells: "## Configure the run", "## Load the catalog", "## Build the
streaming pipeline", "## Inspect the output"
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
gc-anyscale
changed the title
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
/test-template vlm-distillation-catalog-enrichment |
Contributor
|
/test-template |
Contributor
|
/test-template vlm-distillation-catalog-enrichment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.