[Proposal] Add RWKV-7 "Goose" adapter (RWKV7ForCausalLM)

[jlarson4]

Proposal

Add a TransformerBridge adapter for RWKV7ForCausalLM (RWKV-7 "Goose"), an attention-free recurrent architecture with a matrix-valued state and rank-2 delta-rule writes.

Motivation

RWKV-7 has no KV cache and no attention matrix, token mixing is a generalized delta-rule update of a matrix-valued recurrent state. This is a distinct surface from the Mamba/Mamba2 SSM adapters already in the registry (different state structure and update rule), and from every attention model. It opens interp questions about how information is written to and read from the recurrent state over time. It has appeared in recent interp work: WriteSAE applies its recurrent-state SAE method to RWKV-7 alongside Gated DeltaNet and Mamba-2 (its headline causal results are on the latter two, but the method covers RWKV-7) and ships small checkpoints (191M, 0.4B) that are cheap to verify.

Gap scan (2026-06-25): 15 models, ~40K downloads.

Pitch

Implement the RWKV-7 time-mixing (matrix state + delta-rule update) and channel-mixing blocks, exposing the recurrent state read/write as hookable streams.

Loading: RWKV-7 is not native in any transformers release, only RWKV-4 (rwkv) is in CONFIG_MAPPING. The fla-hub / RWKV checkpoints load via remote code (auto_map to modeling_rwkv7, trust_remote_code=True, transformers ≥4.48 for the AutoModel plumbing). openelm.py is the precedent for that remote-code load path. Config markers: model_type=rwkv7, hidden_act=sqrelu, attn=null, low-rank state dims (a/decay/gate/v_low_rank_dim).

Scaffolding: this is not the OpenELM pattern — RWKV-7 is attention-free, so it does not reuse AttentionBridge/MLPBridge. The right precedent is the bridge's recurrent-mixer scaffolding: GatedDeltaNetBridge (used by qwen3_next/qwen3_5) and the SSM mixers (mamba.py for SSMMixerBridge/SSMBlockBridge/DepthwiseConv1DBridge). RWKV-7's generalized delta rule is closely related to Gated DeltaNet, so the time-mixing can extend/parallel GatedDeltaNetBridge rather than start from scratch; add a token-shift/conv and the sqrelu channel-mix MLP.

• Claude Code users can scaffold with /add-model-support fla-hub/rwkv7-0.4B-g1.
• Register at the four sites listed in contributing.md.
• Verify smallest-first: fla-hub/rwkv7-0.1B-g1 (191M), then fla-hub/rwkv7-0.4B-g1, then RWKV/RWKV7-Goose-World3-1.5B-HF.

Additional context

• Paper: https://arxiv.org/abs/2503.14456
• Interp method covering RWKV-7 (recurrent-state SAEs; note its headline causal results are on Gated DeltaNet / Mamba-2, not RWKV-7): https://arxiv.org/abs/2605.12770
• Checkpoint: https://huggingface.co/fla-hub/rwkv7-0.4B-g1
• Found via the hf_scraper architecture-gaps pass (2026-06-25).

Checklist

• I have checked that there is no similar issue in the repo (required)

[mukund1985]

@jlarson4 I'd like to take this one. I've been working through the SSM adapter space — recently shipped NemotronH (#1434) and Zamba2 (#1486), so I have a decent handle on how SSMBlockBridge and the passthrough patterns work. RWKV-7's matrix-valued state and rank-2 delta-rule updates are a different primitive, but the GatedDeltaNetBridge looks like a natural starting point. Happy to branch off dev and work through the weight mapping. Let me know if you want me to go ahead.

[jlarson4]

@mukund1985 feel free to move on to this one after you've got a PR up for #1469