[Feature] Mamba Trainer Integration #348

Closed

clairesonglee wants to merge 35 commits into

mainfrom

clairlee/dev/mamba

Commits on Oct 2, 2025

Disable torchtitan activation checkpointing for better 8B model performance
alfuyao-amd
authored
Disable torchtitan activation checkpointing for better 8B model performance
alfuyao-amd
authored
Set default for MI300 best perf.
alfuyao-amd
authored
Update llama3.1_70B-BF16-pretrain.yaml drop log frequency to reduce log perf. overhead
alfuyao-amd
authored
Update llama3.1_70B-FP8-pretrain.yaml drop log frequency to reduce log perf. overhead
alfuyao-amd
authored
Update llama3.1_8B-BF16-pretrain.yaml drop log frequency to reduce log perf. overhead
alfuyao-amd
authored
Update llama3.1_8B-FP8-pretrain.yaml drop log frequency to reduce log perf. overhead
alfuyao-amd
authored

Commits on Oct 6, 2025

tw script update
clairesonglee
committed

Commits on Oct 8, 2025

remove cluster-specific commands
clairesonglee
authored

Commits on Oct 9, 2025

update common perf arguments - ce fusion - moe gemms
vidushi8
committed

Commits on Nov 19, 2025

Commits on Nov 20, 2025

fix PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32 typo. It will now default to 0
vidushi8
committed

Commits on Nov 21, 2025

Merge branch 'main' into release/v25.10
vidushi8
committed

Commits on Nov 25, 2025

update torch profiler gzip to false
vidushi8
committed
support turbo groupgemm in titan

DCCS-4317
authored and
JohnQinAMD
committed
add turbo fp8 gemm and attn via Converter

llying-001
authored and
JohnQinAMD
committed
update deepseek_v3 config and load balance config

llying-001
authored and
JohnQinAMD
committed
add classic attn but with issues

llying-001
authored and
JohnQinAMD
committed
update classic attention args for deepseek_v3

llying-001
authored and
JohnQinAMD
committed
update config

llying-001
authored and
JohnQinAMD
committed
support new turo fp8 api
JohnQinAMD
committed

Commits on Nov 26, 2025

Commits on Dec 10, 2025

initial commit
clairesonglee
committed

Commits on Dec 16, 2025

set self.lr_warmup_steps < self.lr_decay_steps
clairesonglee
committed

Commits on Dec 18, 2025

unwrap model to remove loss_mask parameter
clairesonglee
committed