Add challenge 108: Cross-Attention (Medium)

[claude]

Summary

• Adds a multi-head cross-attention challenge as challenges/medium/108_cross_attention/
• Fills a real gap: existing attention challenges (6_softmax_attention, 12_multi_head_attention, 53_causal_attention, 59_sliding_window_attn, 80_grouped_query_attention, etc.) are all self-attention or causal variants with matching query and key lengths. Cross-attention is the fundamental building block used by encoder-decoder transformers (T5, BART, Whisper) and by the text-conditioning path of diffusion models like Stable Diffusion
• Layout is (position, head, dim) for all four tensors. Decoder queries of shape (M, H, D) attend to encoder keys/values of shape (N, H, D) with M ≠ N in general, no causal mask, per-head scaled dot-product attention
• Reference impl uses only standard PyTorch ops (matmul + softmax) so it lowers cleanly on both CUDA and XLA
• Functional tests cover: 1×1 edges, decode-like (M=1, N=16), prefill-like (M=4, N=1), zero inputs, uniform mixed values, power-of-2 sizes, non-power-of-2 shapes (30×45, 100×200), and realistic Whisper/BART-scale cases up to (M=128, N=512, H=16, D=64)
• Performance test: M=1024, N=2048, H=16, D=128 (~48 MB, safely within 5× of Tesla T4 VRAM)

Validated end-to-end on the platform with run_challenge.py --language cuda (single-block-per-(query, head), softmax over N with shared-memory reductions): both the example run and the full functional/performance suite pass with Status: success | Output: ✓ All tests passed.

Test plan

• pre-commit run --all-files (black, isort, flake8, clang-format, mojo format)
• python scripts/run_challenge.py challenges/medium/108_cross_attention --language cuda --action run — example test passes
• python scripts/run_challenge.py challenges/medium/108_cross_attention --language cuda --action submit — all functional + performance tests pass on Tesla T4
• CLAUDE.md checklist reviewed: 5 class attrs, no __init__, reference asserts shape+dtype only, all tensors use device=self.device, 11 functional cases spanning edges/powers-of-2/non-powers-of-2/zeros/mixed/realistic, perf fits 5× in 16 GB VRAM
• All 6 starter files present with correct per-framework parameter comment; starters compile but do not solve

🤖 Generated with Claude Code