chore: rebase fork to NVIDIA/NeMo upstream HEAD by JakobHavtorn · Pull Request #2 · corticph/NeMo

Jakob Drachmann Havtorn (JakobHavtorn) · 2026-08-31T13:52:22Z

Summary

Fast-forwards the fork's main to the current HEAD of NVIDIA/NeMo (66f1211), bringing in 397 upstream commits since the fork's base (b90a528, v2.7 era).

This PR contains only upstream commits — no custom changes.

What's included

Key upstream changes since our base:

c6bfb2718 Fix word confidence return (Fix word confidence return NVIDIA-NeMo/Speech#15249)
348c25406 AED greedy confidence estimation (AED greedy confidence estimation NVIDIA-NeMo/Speech#11573)
9bafd37e5 TDT confidence fix (TDT confidence fix NVIDIA-NeMo/Speech#8982)
a5448f3d3 Loop labels greedy decoding: faster implementation ("Loop labels" greedy decoding: faster implementation NVIDIA-NeMo/Speech#8286)
f3aea9b0c ASR Confidence measure → method renames ([ASR] Confidence measure -> method renames NVIDIA-NeMo/Speech#7434)
33100e0ad ASR Confidence update and tutorial (ASR Confidence update and tutorial NVIDIA-NeMo/Speech#6810)
Various TTS, CI, and infrastructure improvements

Why

The fork was ~397 commits behind upstream. This rebases the fork to stay current with NVIDIA's NeMo development, enabling us to benefit from upstream fixes (including confidence/decoding improvements) without maintaining a large divergence.

Custom confidence methods (margin, max_prob_raw, entropy, top3_mass) are tracked in a separate PR (#2).

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* Easy MagpieTTS squashed commit Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * comments in magpietts inference Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * comments in inference.py Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * comments Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * Apply isort and black reformatting Signed-off-by: paarthneekhara <paarthneekhara@users.noreply.github.com> * remove unnecessary comments Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * Apply isort and black reformatting Signed-off-by: paarthneekhara <paarthneekhara@users.noreply.github.com> * address some of subhankar's comments Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * Apply isort and black reformatting Signed-off-by: paarthneekhara <paarthneekhara@users.noreply.github.com> * utmos changes Signed-off-by: Shehzeen Hussain <shehzeensh@gmail.com> * change ... to pass Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * dev train tests and some comments Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * bring back comments to inference file Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * Apply isort and black reformatting Signed-off-by: paarthneekhara <paarthneekhara@users.noreply.github.com> * add dev tests to yaml Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * inference test added Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * Online PO test for EasyMagpie Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * ci cd tests update Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * Apply isort and black reformatting Signed-off-by: paarthneekhara <paarthneekhara@users.noreply.github.com> * add doc for training mode Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * config update Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * increase timeout for PO Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> * remove unnessary line Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> --------- Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> Signed-off-by: paarthneekhara <paarthneekhara@users.noreply.github.com> Signed-off-by: Shehzeen Hussain <shehzeensh@gmail.com> Co-authored-by: paarthneekhara <paarthneekhara@users.noreply.github.com> Co-authored-by: Shehzeen Hussain <shehzeensh@gmail.com>

* Make all TTS tests HUBOFFLINE and deterministic Signed-off-by: Jason <jasoli@nvidia.com> * add pre-saved checkpoint paths for ASR and EOU Signed-off-by: Jason <jasoli@nvidia.com> * tighten bounds Signed-off-by: Jason <jasoli@nvidia.com> --------- Signed-off-by: Jason <jasoli@nvidia.com>

…5522) * add streaming inference support for unified rnnt model Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * Apply isort and black reformatting Signed-off-by: andrusenkoau <andrusenkoau@users.noreply.github.com> * minor fixes Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * fixes Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * minor fixes Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> --------- Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> Signed-off-by: andrusenkoau <andrusenkoau@users.noreply.github.com> Co-authored-by: andrusenkoau <andrusenkoau@users.noreply.github.com>

If the configured context length is too short to fit commonly used text contexts we detect that and error out. Signed-off-by: Fejgin, Roy <rfejgin@nvidia.com>

* Add MagpieTTS finetuning docs Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com> * Finetuning docs review changes Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com> * ci: Update docs build job to exclude cu12 extra (NVIDIA-NeMo#15553) * Test no extras docs build Signed-off-by: Charlie Truong <chtruong@nvidia.com> * ci: Update docs job to use 0.83.0 templates Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Uncomment push cases for build-docs github action Signed-off-by: Charlie Truong <chtruong@nvidia.com> --------- Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Rename index for attention prior weights (NVIDIA-NeMo#15551) Signed-off-by: Subhankar Ghosh <subhankarg@nvidia.com> * Ignore PnC for WER calculation: streaming ASR inference (NVIDIA-NeMo#15550) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * ci: upgrade GitHub Actions for Node.js 24 compatibility (NVIDIA-NeMo#15537) Upgrades actions to versions compatible with the Node.js 24 runtime: - actions/checkout: → v6 - actions/upload-artifact: → v6 - actions/download-artifact: → v7 - actions/github-script: → v8 - actions/setup-python: → v6 Mirrors: NVIDIA/Megatron-LM@1d5e68b Signed-off-by: oliver könig <okoenig@nvidia.com> * Add VoiceChat to README (NVIDIA-NeMo#15547) * Update README.md Signed-off-by: Jason <jasoli@nvidia.com> * Revise Nemotron VoiceChat release details in README Updated the release information for Nemotron VoiceChat and added details about its features and early access. Signed-off-by: zhehuaichen <139396994+zhehuaichen@users.noreply.github.com> --------- Signed-off-by: Jason <jasoli@nvidia.com> Signed-off-by: zhehuaichen <139396994+zhehuaichen@users.noreply.github.com> Co-authored-by: zhehuaichen <139396994+zhehuaichen@users.noreply.github.com> * Add ASR-EOU models and training/eval scripts (NVIDIA-NeMo#14740) * initial commit for end-of-utterance detection Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> * change targets to long() type Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> * change output_types() Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> * add random padding and refactor for multiple utterances per sample Signed-off-by: stevehuang52 <heh@nvidia.com> * add handling multiple text groundtruth Signed-off-by: stevehuang52 <heh@nvidia.com> * update and add eval scripts Signed-off-by: stevehuang52 <heh@nvidia.com> * drop sou label and add eob label Signed-off-by: stevehuang52 <heh@nvidia.com> * update hybrid-rnnt-ctc and rnnt models to use eou dataset Signed-off-by: stevehuang52 <heh@nvidia.com> * set default return eou frame label to false Signed-off-by: stevehuang52 <heh@nvidia.com> * handle empty utterance Signed-off-by: stevehuang52 <heh@nvidia.com> * add script for injecting special eou tokens into SPE tokenizer Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor eou eval utils Signed-off-by: stevehuang52 <heh@nvidia.com> * add eou rnnt training Signed-off-by: stevehuang52 <heh@nvidia.com> * update doc Signed-off-by: stevehuang52 <heh@nvidia.com> * update data augmentation Signed-off-by: stevehuang52 <heh@nvidia.com> * update data related functions Signed-off-by: stevehuang52 <heh@nvidia.com> * fix tokenizer with eou tokens Signed-off-by: stevehuang52 <heh@nvidia.com> * adding eou force aligner Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> * update for eou Signed-off-by: stevehuang52 <heh@nvidia.com> * fix the case when 'segments_level_ctm_filepath' is not produced Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> * fix force aligner Signed-off-by: stevehuang52 <heh@nvidia.com> * fix aligner Signed-off-by: stevehuang52 <heh@nvidia.com> * update for asr-eou Signed-off-by: stevehuang52 <heh@nvidia.com> * clean up and update infer Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * fix rnnt_decoding for empty string Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update padding augment Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * fix eob metric logging Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor and add hybrid model Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update EOU models Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor percentile calculation Signed-off-by: stevehuang52 <heh@nvidia.com> * update augmentation Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update model and cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update frame eou Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * add adapter to eou Signed-off-by: stevehuang52 <heh@nvidia.com> * remove pdb Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * add cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * fix eou metric Signed-off-by: stevehuang52 <heh@nvidia.com> * update adapter Signed-off-by: stevehuang52 <heh@nvidia.com> * add scripts Signed-off-by: stevehuang52 <heh@nvidia.com> * update docstring Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update generate eval data Signed-off-by: stevehuang52 <heh@nvidia.com> * update eou val Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * add drop_pnc=true as default for dataloading Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * fix miss rate Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * add ignore_eob_label Signed-off-by: stevehuang52 <heh@nvidia.com> * fix and update Signed-off-by: stevehuang52 <heh@nvidia.com> * improve lhotse augmentation Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * add debug info Signed-off-by: stevehuang52 <heh@nvidia.com> * improve data augmentation Signed-off-by: stevehuang52 <heh@nvidia.com> * update utils Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update dataloader Signed-off-by: stevehuang52 <heh@nvidia.com> * update oomptimizer Signed-off-by: stevehuang52 <heh@nvidia.com> * update oomptimizer Signed-off-by: stevehuang52 <heh@nvidia.com> * update eou model Signed-off-by: stevehuang52 <heh@nvidia.com> * update eou model Signed-off-by: stevehuang52 <heh@nvidia.com> * update eou model Signed-off-by: stevehuang52 <heh@nvidia.com> * update augmentation Signed-off-by: stevehuang52 <heh@nvidia.com> * update aug Signed-off-by: stevehuang52 <heh@nvidia.com> * update augment Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update drop pnc func Signed-off-by: stevehuang52 <heh@nvidia.com> * update eou finetune Signed-off-by: stevehuang52 <heh@nvidia.com> * update transcribe Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * fix cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * clean up for PR Signed-off-by: stevehuang52 <heh@nvidia.com> * clean up Signed-off-by: stevehuang52 <heh@nvidia.com> * Potential fix for code scanning alert no. 16191: Explicit returns mixed with implicit (fall through) returns Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Potential fix for code scanning alert no. 16190: Explicit returns mixed with implicit (fall through) returns Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Apply isort and black reformatting Signed-off-by: stevehuang52 <stevehuang52@users.noreply.github.com> * Potential fix for code scanning alert no. 16185: File is not always closed Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * clean up Signed-off-by: stevehuang52 <heh@nvidia.com> * clean up Signed-off-by: stevehuang52 <heh@nvidia.com> * fix pylint&flake8 Signed-off-by: stevehuang52 <heh@nvidia.com> * fix pylint Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor Signed-off-by: stevehuang52 <heh@nvidia.com> * update pr Signed-off-by: stevehuang52 <heh@nvidia.com> * update adapter Signed-off-by: stevehuang52 <heh@nvidia.com> * clean up Signed-off-by: stevehuang52 <heh@nvidia.com> * update readme, test, etc Signed-off-by: He Huang <heh@nvidia.com> * Apply isort and black reformatting Signed-off-by: stevehuang52 <stevehuang52@users.noreply.github.com> * update doc Signed-off-by: He Huang <heh@nvidia.com> * clean up Signed-off-by: He Huang <heh@nvidia.com> * fix and rename Signed-off-by: He Huang <heh@nvidia.com> * update doc Signed-off-by: He Huang <heh@nvidia.com> * clean up Signed-off-by: He Huang <heh@nvidia.com> * move all length aug to invalid Signed-off-by: He Huang <heh@nvidia.com> * fix typo Signed-off-by: He Huang <heh@nvidia.com> * rename and move to scripts/asr_eou Signed-off-by: He Huang <heh@nvidia.com> * fix ci Signed-off-by: He Huang <heh@nvidia.com> * fix ci Signed-off-by: He Huang <heh@nvidia.com> * clean up Signed-off-by: He Huang <heh@nvidia.com> * clean up Signed-off-by: He Huang <heh@nvidia.com> * fix linting Signed-off-by: He Huang <heh@nvidia.com> * fix ci Signed-off-by: He Huang <heh@nvidia.com> * Apply isort and black reformatting Signed-off-by: stevehuang52 <stevehuang52@users.noreply.github.com> * Potential fix for code scanning alert no. 17270: Explicit export is not defined Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Potential fix for code scanning alert no. 17271: Explicit export is not defined Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Potential fix for code scanning alert no. 17272: Explicit export is not defined Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> --------- Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> Signed-off-by: stevehuang52 <heh@nvidia.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: stevehuang52 <stevehuang52@users.noreply.github.com> Signed-off-by: He Huang <heh@nvidia.com> Co-authored-by: Weiqing Wang <weiqingw@nvidia.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: stevehuang52 <stevehuang52@users.noreply.github.com> * Fix freesound url Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com> * Fix freesound url Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com> --------- Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com> Signed-off-by: Charlie Truong <chtruong@nvidia.com> Signed-off-by: Subhankar Ghosh <subhankarg@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: oliver könig <okoenig@nvidia.com> Signed-off-by: Jason <jasoli@nvidia.com> Signed-off-by: zhehuaichen <139396994+zhehuaichen@users.noreply.github.com> Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> Signed-off-by: stevehuang52 <heh@nvidia.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: stevehuang52 <stevehuang52@users.noreply.github.com> Signed-off-by: He Huang <heh@nvidia.com> Co-authored-by: Charlie Truong <chtruong@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Jason <jasoli@nvidia.com> Co-authored-by: zhehuaichen <139396994+zhehuaichen@users.noreply.github.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: Weiqing Wang <weiqingw@nvidia.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: stevehuang52 <stevehuang52@users.noreply.github.com>

* Adding the new fixed cpWER that is identical to meeteval Signed-off-by: taejinp <tango4j@gmail.com> * Finalize cpwer fix Signed-off-by: taejinp <tango4j@gmail.com> * Apply isort and black reformatting Signed-off-by: tango4j <tango4j@users.noreply.github.com> * Adding fixed versions Signed-off-by: taejinp <tango4j@gmail.com> * Apply isort and black reformatting Signed-off-by: tango4j <tango4j@users.noreply.github.com> --------- Signed-off-by: taejinp <tango4j@gmail.com> Signed-off-by: tango4j <tango4j@users.noreply.github.com> Co-authored-by: tango4j <tango4j@users.noreply.github.com>

* minor fixies Signed-off-by: naymaraq <dkaramyan@nvidia.com> * Apply isort and black reformatting Signed-off-by: naymaraq <naymaraq@users.noreply.github.com> --------- Signed-off-by: naymaraq <dkaramyan@nvidia.com> Signed-off-by: naymaraq <naymaraq@users.noreply.github.com> Co-authored-by: naymaraq <dkaramyan@nvidia.com> Co-authored-by: naymaraq <naymaraq@users.noreply.github.com>

* [TTS] Add code for training semantic codec Signed-off-by: Ryan <rlangman@nvidia.com> * Apply isort and black reformatting Signed-off-by: rlangman <rlangman@users.noreply.github.com> * Rename slm decoder to predictor Signed-off-by: Ryan <rlangman@nvidia.com> * Add audio codec model tests Signed-off-by: Ryan <rlangman@nvidia.com> * Add HF_HUB_OFFLINE to audio codec tests Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: rlangman <rlangman@users.noreply.github.com> Co-authored-by: rlangman <rlangman@users.noreply.github.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>

Signed-off-by: arushid <arushid@nvidia.com> Co-authored-by: Dav Karamyan <47416614+naymaraq@users.noreply.github.com>

* remove enable PnC flag Signed-off-by: naymaraq <dkaramyan@nvidia.com> * bugfix Signed-off-by: naymaraq <dkaramyan@nvidia.com> * fix tutorial Signed-off-by: naymaraq <dkaramyan@nvidia.com> * rm pnc line from usage example Signed-off-by: naymaraq <dkaramyan@nvidia.com> * add functional tests for cache aware models Signed-off-by: naymaraq <dkaramyan@nvidia.com> * Apply isort and black reformatting Signed-off-by: naymaraq <naymaraq@users.noreply.github.com> * rename augment_with_default -> fill_defaults Signed-off-by: naymaraq <dkaramyan@nvidia.com> * rm dead code Signed-off-by: naymaraq <dkaramyan@nvidia.com> * Apply isort and black reformatting Signed-off-by: naymaraq <naymaraq@users.noreply.github.com> * set enable pnc to false Signed-off-by: naymaraq <dkaramyan@nvidia.com> --------- Signed-off-by: naymaraq <dkaramyan@nvidia.com> Co-authored-by: naymaraq <dkaramyan@nvidia.com>

Signed-off-by: naymaraq <dkaramyan@nvidia.com> Co-authored-by: naymaraq <dkaramyan@nvidia.com>

Signed-off-by: aandrusenko <aandrusenko@nvidia.com>

Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>

Signed-off-by: L0RD-ZER0 <68327382+L0RD-ZER0@users.noreply.github.com>

…15610) Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>

…IA-NeMo#15612) - add /debug-training-logs slash command that analyzes SLURM worker stderr logs and optional AIStore daemon logs to find root causes of distributed training failures - covers NCCL timeout analysis: distinguishes straggler ranks (stuck in data loading) from GPU fabric hangs by comparing enqueued vs completed work across ALL ranks - includes AIStore log parsing: file time ranges, timezone verification, error counter tracking, proxy/target correlation - documents NeMo-specific sync points (PreemptionCallback broadcast, checkpoint broadcasts, DDP allreduce) that can cause rank desync - documents Lhotse data loading pitfalls: missing read timeouts, m4a BytesIO extension loss, idle connection resets, fault_tolerant silent drops - includes instructions to obtain logs via scp and download AIS daemon logs via ais CLI with env var auth Signed-off-by: Abhishek Gaikwad <gaikwadabhishek1997@gmail.com>

Signed-off-by: Ryan <rlangman@nvidia.com>

…ducibility, speed up training and support half precision inference (NVIDIA-NeMo#15583) * Add new formatter and force codec to runs in the same device of TTS model Signed-off-by: Edresson Casanova <edresson1@gmail.com> * Set find_unused_parameters=False as default to speedup training Signed-off-by: Edresson Casanova <edresson1@gmail.com> * Apply isort and black reformatting Signed-off-by: Edresson <Edresson@users.noreply.github.com> * Fix model input data type Signed-off-by: Edresson Casanova <edresson1@gmail.com> * Apply isort and black reformatting Signed-off-by: Edresson <Edresson@users.noreply.github.com> * Add support for half precision inference Signed-off-by: Edresson Casanova <edresson1@gmail.com> * Apply isort and black reformatting Signed-off-by: Edresson <Edresson@users.noreply.github.com> * Fix RMSnorm Signed-off-by: Edresson Casanova <edresson1@gmail.com> * Update debug_dtype default parameter Signed-off-by: Edresson Casanova <edresson1@gmail.com> * Apply isort and black reformatting Signed-off-by: Edresson <Edresson@users.noreply.github.com> * Add unit tests for duplex formatters Signed-off-by: Edresson Casanova <edresson1@gmail.com> * Apply isort and black reformatting Signed-off-by: Edresson <Edresson@users.noreply.github.com> * Replace fastcopy with deepcopy Signed-off-by: Edresson Casanova <edresson1@gmail.com> * Apply isort and black reformatting Signed-off-by: Edresson <Edresson@users.noreply.github.com> * Remove unecessary comments Signed-off-by: Edresson Casanova <edresson1@gmail.com> * Update unit tests Signed-off-by: Edresson Casanova <edresson1@gmail.com> * Add function for dtype debug and create a method to check codec precision Signed-off-by: Edresson Casanova <edresson1@gmail.com> * Apply isort and black reformatting Signed-off-by: Edresson <Edresson@users.noreply.github.com> --------- Signed-off-by: Edresson Casanova <edresson1@gmail.com> Signed-off-by: Edresson <Edresson@users.noreply.github.com> Co-authored-by: Edresson <Edresson@users.noreply.github.com>

* cleunup tail padding algorithms Signed-off-by: naymaraq <dkaramyan@nvidia.com> * remove tail margin algorithms at all Signed-off-by: naymaraq <dkaramyan@nvidia.com> * Apply isort and black reformatting Signed-off-by: naymaraq <naymaraq@users.noreply.github.com> --------- Signed-off-by: naymaraq <dkaramyan@nvidia.com> Co-authored-by: naymaraq <dkaramyan@nvidia.com>

NVIDIA-NeMo#15447) * WIP: bringing Yifan's changes to main Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Add workaround for exp_manager issue Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Support reading indexed JSONL datasets with ShareGPT format Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Support reading indexed tarred datasets with ShareGPT format Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Refactor for compactness Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fixes for real-life data Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fixes for real-life data Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fixes for real-life data Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fixes for missing wids-meta.json Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fixes for tarfile edge cases Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fixes for real-world tar files Signed-off-by: Piotr Żelasko <petezor@gmail.com> * move salm llm init to configure_model Signed-off-by: Piotr Żelasko <petezor@gmail.com> * fix: delayed perception init Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Add AutomodelParallelStrategy for Automodel LLM support Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Replace HF Automodel with NeMo Automodel for SALM's LLM backbone Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Update salm default config with new options Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Init fixes Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fix dtype initialization Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fix mesh selection for speech encoder Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fix for mismatched device_mesh axis names in gradient clipping - use automodel's utility Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fix for using embed_tokens in FSDP context before running forward on full LLM Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Definitive fix for using embed_tokens outside of llm with fsdp Signed-off-by: Piotr Żelasko <petezor@gmail.com> * this version actually works with Automodel Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * fix from_pretrained with transformers v5 Signed-off-by: Piotr Żelasko <petezor@gmail.com> * fix from_pretrained with transformers v5 Signed-off-by: Piotr Żelasko <petezor@gmail.com> * fix generate/eval Signed-off-by: Piotr Żelasko <petezor@gmail.com> * fix to_hf Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fixes for AutoTokenizer decoding in v5 Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Flag to run configure_model() at the end of __init__ for safetensors converted models Signed-off-by: Piotr Żelasko <petezor@gmail.com> * preliminary: support distributed models in to_hf.py Signed-off-by: Piotr Żelasko <petezor@gmail.com> * fix passing automodel kwargs Signed-off-by: Piotr Żelasko <petezor@gmail.com> * fix Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Enable inference with model parallelism Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fix for lightning save_hyperparameters() call Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fix for loading into DTensor Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Accelerate loading DTensor Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Accelerate loading DTensor Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Accelerate loading DTensor Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fix for pe buffers not in ckpt (essentially strict=False) Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Add Nemotron Nano v3 prompt formatter with <think> reasoning support Implements NemotronNanoV3PromptFormatter (NAME="nemotron-nano-v3") using ChatML-style <|im_start|>/<|im_end|> template with encode_dialog override that handles: auto-insert empty system turn, history thinking truncation, <think></think> prepend for non-thinking assistant turns, and dynamic inference prefix (thinking on/off). Includes Lhotse Cut integration via registered_prompt_format_fn. Verified against HF apply_chat_template for nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (both string and token match). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Automodel LoRA support Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fixes for model parallel Signed-off-by: Piotr Żelasko <petezor@gmail.com> * LoRA fix Signed-off-by: Piotr Żelasko <petezor@gmail.com> * small ckpt conversion/inference fix Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Separate SALM and SALMAutomodel into independent classes Restore salm.py to its original HF Transformers + PEFT LoRA implementation from main, and extract the NeMo Automodel-based implementation into a new SALMAutomodel class in salm_automodel.py. This keeps both backends available and independent, with scripts auto-detecting the model class from config.json. - salm.py: restored from main (eager init, HF PEFT, move_embedding) - salm_automodel.py: new file with SALMAutomodel (deferred init, automodel LoRA) - salm_train.py: selects model class via model.use_nemo_automodel config key - salm_eval.py/salm_generate.py: auto-detect model class from config.json - salm_automodel.yaml: new config for SALMAutomodel training - Tests split into test_salm.py (CPU) and test_salm_automodel.py (CUDA) - New functional test SPEECHLM_Automodel_Training_SALM.sh Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> * fix linters Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Add SALMAutomodel docs and speechlm2 pip extra Add documentation for SALMAutomodel (NeMo Automodel variant of SALM) across all speechlm2 doc pages: intro, models, configs, and training_and_scaling. Create pip install nemo-toolkit[speechlm2] extra that composes speechlm2-only (nemo_automodel git dep) + asr + tts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add SALMAutomodel tutorial notebook and fix EP/FSDP2 docs Add tutorials/speechlm2/SpeechLM_With_NeMo_Automodel.ipynb covering the full pipeline: data download, training, checkpoint conversion, and evaluation with Nemotron Nano V3 MoE backbone on 2 GPUs. Fix docs to clarify that Expert Parallelism reuses the FSDP2 data-parallel axis — dense layers are sharded via FSDP2 while MoE layers use EP on the same GPUs, not a separate dimension. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix uv torch index conflict for speechlm2 extra The docs CI runs `uv sync --all-extras --all-groups` which resolves the speechlm2 extra pulling nemo_automodel from git. uv treats git source deps as workspace members and applies their [tool.uv.sources], causing a conflict: Automodel maps torch to per-platform indexes while NeMo defaulted to PyPI for all platforms. Add matching [tool.uv.sources] for torch to pyproject.toml and regenerate uv.lock with nemo_automodel included. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Remove direction arg Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Apply isort and black reformatting Signed-off-by: pzelasko <pzelasko@users.noreply.github.com> * fix linter Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * fix tests Signed-off-by: Piotr Żelasko <petezor@gmail.com> * fixes Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * fixes for trust_remote_code Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Apply isort and black reformatting Signed-off-by: pzelasko <pzelasko@users.noreply.github.com> * Add explicit enable_thinking support to SALM eval paths * Apply isort and black reformatting Signed-off-by: pzelasko <pzelasko@users.noreply.github.com> * fix inference with ep_size=1 for automodel models Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Fixes Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Fixes for inference and tutorial Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Apply isort and black reformatting Signed-off-by: pzelasko <pzelasko@users.noreply.github.com> * Remove deprecated activation_checkpointing parameter everywhere Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fix CI Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Fix to_hf.py crash when run without torchrun Guard dist.init_process_group on RANK env var presence so the script works with plain `python` (single-file checkpoints) as well as `torchrun` (distributed checkpoints). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Apply suggestions from code review Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Add flashoptim support and bf16-automodel half precision setup Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Apply isort and black reformatting Signed-off-by: pzelasko <pzelasko@users.noreply.github.com> * patch flashoptim handling of unevenly sharded state dicts Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Reproducibility fix Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Refactor AutomodelPrecision to FlashPrecision to enable re-use by other collections in subsequent PRs Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Apply isort and black reformatting Signed-off-by: pzelasko <pzelasko@users.noreply.github.com> * Address code review Signed-off-by: Piotr Żelasko <petezor@gmail.com> * disable linter Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fix test Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * fix for torch.compile config Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Apply isort and black reformatting Signed-off-by: pzelasko <pzelasko@users.noreply.github.com> * fix tests Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Dataloader DP rank patch for Automodel's device_mesh Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Apply isort and black reformatting Signed-off-by: pzelasko <pzelasko@users.noreply.github.com> * fix sloppy fix Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * fix CI HF tokenizer download issue Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Add tests for correct DP rank resolution in the dataloader Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Apply isort and black reformatting Signed-off-by: pzelasko <pzelasko@users.noreply.github.com> * xfail tests with corrupted tokenizer in CI Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Update test pytorch version safeguard Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fix new peft version requiring newer torchao than available in CI container Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fixes Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> * Bump Automodel pin for transformers compat Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> --------- Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: pzelasko <pzelasko@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: pzelasko <pzelasko@users.noreply.github.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

…sor (NVIDIA-NeMo#15613) * Add CLAUDE.md and symlinks for Codex/Cursor Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Address review + babysit PR action + 2 skills (babysit-pr, fix-issue) Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fix double-posting of 'taking over' comment and change the text Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Fix truncated sentence Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Make babysitter less autonomous Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Harden babysitter: speech_team-only, fork guards, drop reaction path Addresses Charlie's review on NVIDIA-NeMo#15613. Adds per-job preflights that verify the acting user (label sender / approval commenter / execute-fix sender) is an active NVIDIA-NeMo/speech_team member, and fork-guards every PR-scoped job so the babysitter never runs on forks. Also blocks manual-label bypass by requiring a bot-authored plan comment before execute-fix proceeds, and drops the cron+reaction approval path in favor of natural-language replies. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…VIDIA-NeMo#15619) Bumps [protobufjs](https://github.com/protobufjs/protobuf.js) from 7.5.3 to 7.5.5. - [Release notes](https://github.com/protobufjs/protobuf.js/releases) - [Changelog](https://github.com/protobufjs/protobuf.js/blob/master/CHANGELOG.md) - [Commits](protobufjs/protobuf.js@protobufjs-v7.5.3...protobufjs-v7.5.5) --- updated-dependencies: - dependency-name: protobufjs dependency-version: 7.5.5 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [mako](https://github.com/sqlalchemy/mako) from 1.3.10 to 1.3.11. - [Release notes](https://github.com/sqlalchemy/mako/releases) - [Changelog](https://github.com/sqlalchemy/mako/blob/main/CHANGES) - [Commits](https://github.com/sqlalchemy/mako/commits) --- updated-dependencies: - dependency-name: mako dependency-version: 1.3.11 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [pillow](https://github.com/python-pillow/Pillow) from 12.1.1 to 12.2.0. - [Release notes](https://github.com/python-pillow/Pillow/releases) - [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst) - [Commits](python-pillow/Pillow@12.1.1...12.2.0) --- updated-dependencies: - dependency-name: pillow dependency-version: 12.2.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Signed-off-by: Edresson Casanova <ecasanova@nvidia.com>

…dy decoding (NVIDIA-NeMo#16098) `GreedyBatchedTDTLabelLoopingComputer.torch_impl` rebinds `durations` for the whole batch on every iteration of the stage 1.2 inner loop, while `labels` and `scores` are written back only under `advance_mask`. Utterances that already found their non-blank label therefore have their predicted duration overwritten by a joint evaluation taken at a later encoder frame, and stage 2 stores that stale value as `token_duration`. Since `RNNTDecoding._compute_offsets_tdt` derives `end_offset` from `start_offset + token_duration`, and `compute_timestamps=True` enables `tdt_include_token_duration` automatically, this makes TDT end timestamps depend on which other utterances share the batch. The inner loop only iterates while some other utterance is still searching, which is what ties the corruption to batch composition. Apply the same guard the CUDA graph implementation of this algorithm already uses in `_inner_loop_step_find_next_non_blank`, so both implementations agree. Affects the `torch_impl` path only: CPU decoding, `use_cuda_graph_decoder=False`, missing CUDA conditional-node support, and validation decoding with CUDA graphs disabled. Signed-off-by: Udaya Tejas <udayatejas2004@gmail.com>

…VIDIA-NeMo#16096) `GreedyTDTInfer._greedy_decode` advanced `time_idx` by one extra frame whenever the per-frame symbol budget was exhausted, not only when the budget was exhausted on a zero-duration prediction. With `max_symbols_per_step=1` the budget is exhausted on every frame, so the decoder skipped `duration + 1` frames every time: tokens were dropped and the surviving timestamps were late, while `token_duration` still reported the predicted duration, making the hypothesis self-contradictory. The inner loop can only exit with `skip == 0` because the budget ran out, so the forced advance belongs in the existing zero-duration branch and the trailing conditional can go. This also keeps the `preserve_alignments` / `preserve_frame_confidence` buffers, which are extended by `skip` entries, in step with `time_idx`. With random weights over 468 single-vs-batched utterance comparisons, `GreedyTDTInfer` disagreed with `GreedyBatchedTDTInfer` on 135 of them before the change and on none after it. Signed-off-by: Udaya Tejas <udayatejas2004@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com>

…NeMo#16065) * add back changes Signed-off-by: Jason <jasoli@nvidia.com> * add debug print Signed-off-by: Jason <jasoli@nvidia.com> * Update common.py Signed-off-by: Jason <jasoli@nvidia.com> --------- Signed-off-by: Jason <jasoli@nvidia.com>

Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>

…#16117) * add phoneme control augmentation to multiturn dataloader Signed-off-by: paarthneekhara <paarth.n@gmail.com> * Apply suggestions from code review Co-authored-by: Jason <jasoli@nvidia.com> Signed-off-by: Jason <jasoli@nvidia.com> --------- Signed-off-by: paarthneekhara <paarth.n@gmail.com> Signed-off-by: Jason <jasoli@nvidia.com> Co-authored-by: Jason <jasoli@nvidia.com>

…VIDIA-NeMo#16078) `get_SegLST_from_frame_labels` computed each segment's end time as `start_time + frame_len_in_secs * i`, adding the running `start_time` (the previous utterance's end) to an offset already derived from the absolute frame index `i`. With more than one EOU frame, every segment end drifts further from the true EOU time, so `evaluate_eou` compares misplaced boundaries and reports latencies that grow with the number of utterances in the clip. This corrupts `val_eou_latency_p50/p90/p95`, which are used for checkpoint selection. Frame `i` is at absolute time `frame_len_in_secs * i`, so the end time must not include `start_time`; `start_time` keeps tracking the previous utterance's end so segments stay contiguous. The existing unit test only ever set a single frame to 1, where `start_time` is still 0.0 and the bug is invisible. Added regression tests with multiple EOU frames covering both the SegLST builder and the end-to-end metrics. Signed-off-by: Udaya Tejas <udayatejas2004@gmail.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com>

Signed-off-by: Mrudula-itsjuzme <mrudulasankar2007@gmail.com>

* Update torch dependency version to 2.7.0 Signed-off-by: Mahmoud Ashraf <hassouna97.ma@gmail.com> * Update torch version requirements in uv.lock Signed-off-by: Mahmoud Ashraf <hassouna97.ma@gmail.com> --------- Signed-off-by: Mahmoud Ashraf <hassouna97.ma@gmail.com>

…6043) * add prompt/qwen reasoning support Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * minor fixes Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * clean up Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * added simulstream backend Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * isort and black Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * remove system prompt Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * clean up Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * qwen reasoning system prompt cleanup Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * minor fix Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * black formatting Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * clean up Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * clean up WER calculation Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * fix comment Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * minor fix Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * move partial translations to simulstream pipeline Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * clean up/fixes Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * clean up/fixes Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * add Riva prompt Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> * added guarded finalization Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com> --------- Signed-off-by: lilithgrigoryan <lgrigoryan@nvidia.com>

Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>

* test(asr): cover UTF-8 transcription JSON output Signed-off-by: Sylvester Kaczmarek <assistant@SylvesterKaczmarek.com> Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com> * fix(asr): preserve Unicode in transcription JSON Signed-off-by: Sylvester Kaczmarek <assistant@SylvesterKaczmarek.com> Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com> --------- Signed-off-by: Sylvester Kaczmarek <assistant@SylvesterKaczmarek.com> Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>

* Implement per phrase word boosting * Update tests * Document per-phrase boosting params Signed-off-by: Aadam Haq <aadamhaq@sky.com> --------- Signed-off-by: Aadam Haq <aadamhaq@sky.com>

Copilot

Pull request overview

This PR fast-forwards the fork’s main branch to the current upstream NVIDIA-NeMo/Speech HEAD, incorporating a large set of upstream changes across tutorials, tools, scripts, tests, docs, and CI metadata.

Changes:

Update many docs/tutorials and tooling references from NVIDIA/NeMo to NVIDIA-NeMo/Speech, including installation snippets and links.
Refresh assorted utilities and scripts (e.g., safer tar extraction, safer subprocess usage, minor API/typing adjustments).
Update test suite and CI-related files to match upstream structure and dependencies.

Reviewed changes

Copilot reviewed 178 out of 3080 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
nemo/agents/voice_agent/utils/config_manager.py	Voice-agent STT parameter computation update (buffer sizing).
nemo/agents/voice_agent/pipecat/services/nemo/llm.py	Voice-agent LLM initialization path updates (adds config logging).
scripts/installers/Dockerfile.ngramtools	Installer documentation/link updates for the renamed upstream repo.
examples/asr/speech_to_text_eval.py	Uses `transcription_cfg.output_filename` for manifest output path.
tools/nemo_forced_aligner/utils/data_prep.py	Normalizes manifest audio paths via `get_full_path`.
scripts/dataset_processing/tts/hifitts/get_data.py	Uses `safe_extract` for tar extraction.
scripts/dataset_processing/process_vad_data.py	Uses `safe_extract` for tar extraction.
scripts/dataset_processing/process_speech_commands_data.py	Uses `safe_extract` for tar extraction.
scripts/dataset_processing/process_aishell2_data.py	Avoids `shell=True` for `soxi` subprocess calls.
scripts/dataset_processing/get_demand_data.py	Avoids `shell=True` for `soxi` subprocess calls.
tutorials//.ipynb	Updates NeMo install commands to the `NVIDIA-NeMo/Speech` repo form.
.github/workflows/*	Pins/updates workflow action refs and upstream template SHAs.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+        audio_chunk_size_in_secs = self.server_config.stt.get("audio_chunk_size_in_secs", 0.08)
+        buffer_size = audio_chunk_size_in_secs // self.RAW_AUDIO_FRAME_LEN_IN_SECS
        self.stt_params = NeMoSTTInputParams(


@@ -672,6 +672,8 @@ def get_llm_service_from_config(config: DictConfig) -> OpenAILLMService:
    """Get an LLM service from the configuration."""
    backend = config.type

+    logger.info(f"Initializing LLM service from config: {config}")
+


 # How to use? Build it from NeMo root folder:
-# 1. git clone https://github.com/NVIDIA/NeMo.git && cd NeMo
+# 1. git clone https://github.com/NVIDIA-NeMo/Speech.git && cd NeMo
 # 2. DOCKER_BUILDKIT=1 docker build -t nemo:23.03.1 -f ./scripts/installers/Dockerfile.ngramtools .


Jakob Drachmann Havtorn (JakobHavtorn) · 2026-08-31T14:03:59Z

Superseded by #3 — this PR now includes all 16 custom Corti commits (14 old + 2 new) rebased onto upstream HEAD.

Paarth Neekhara (paarthneekhara) and others added 30 commits March 31, 2026 19:09

bring back zeroshot disabling code (NVIDIA-NeMo#15564)

6a0ecb7

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

Fix Qwen3 SALM LoRA init (NVIDIA-NeMo#15570)

b1235c8

Permit text context with frame stacking (NVIDIA-NeMo#15585)

cae54b3

If the configured context length is too short to fit commonly used text contexts we detect that and error out. Signed-off-by: Fejgin, Roy <rfejgin@nvidia.com>

[TTS][Magpietts] Added CFG distillation (NVIDIA-NeMo#15568)

10e1597

docs: Fix docs link in README (NVIDIA-NeMo#15594)

e990600

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

README.md: next and last stable release info (NVIDIA-NeMo#15602)

a0a76d4

Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>

Mask padded frames after feature normalization (NVIDIA-NeMo#15562)

f2fdc29

Signed-off-by: arushid <arushid@nvidia.com> Co-authored-by: Dav Karamyan <47416614+naymaraq@users.noreply.github.com>

fix use_feat_cache feature (NVIDIA-NeMo#15600)

0f6f9aa

Signed-off-by: naymaraq <dkaramyan@nvidia.com> Co-authored-by: naymaraq <dkaramyan@nvidia.com>

add a unified model link to readme (NVIDIA-NeMo#15604)

69288f2

Signed-off-by: aandrusenko <aandrusenko@nvidia.com>

README: refer to "vanilla" 26.02 container (NVIDIA-NeMo#15606)

c9e6926

Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>

fix: typo in readme (NVIDIA-NeMo#15539)

16f469b

Signed-off-by: L0RD-ZER0 <68327382+L0RD-ZER0@users.noreply.github.com>

Limit to peft<=0.18.0 until CI environment is updated (NVIDIA-NeMo#…

fe524ac

…15610) Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>

Fix codec e2e tests (NVIDIA-NeMo#15607)

8dac920

Signed-off-by: Ryan <rlangman@nvidia.com>

[TTS] Added TTS comparison report (NVIDIA-NeMo#15621)

0b7d8c5

Edresson Casanova (Edresson) and others added 15 commits August 21, 2026 22:31

Make reverse-role formatter pickle safe (NVIDIA-NeMo#16123)

ebe62ed

Signed-off-by: Edresson Casanova <ecasanova@nvidia.com>

Fix SpeechLM2 remote code opt-in (NVIDIA-NeMo#16126)

3998328

Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>

[TTS] Updated TTS comparison report (NVIDIA-NeMo#16127)

837a31f

docs: fix minor typos in documentation (NVIDIA-NeMo#16130)

b17490a

Signed-off-by: Mrudula-itsjuzme <mrudulasankar2007@gmail.com>

ci: trigger approval queue when CICD completes (NVIDIA-NeMo#16152)

c85dd64

Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>

fix(ci): serialize approval queue runs (NVIDIA-NeMo#16153)

6281c93

Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>

[ASR] Wordboosting by phrase (NVIDIA-NeMo#15923)

66f1211

* Implement per phrase word boosting * Update tests * Document per-phrase boosting params Signed-off-by: Aadam Haq <aadamhaq@sky.com> --------- Signed-off-by: Aadam Haq <aadamhaq@sky.com>

Copilot AI lite review requested due to automatic review settings August 31, 2026 13:52

Copilot started reviewing on behalf of Jakob Drachmann Havtorn (JakobHavtorn) August 31, 2026 13:52 View session

github-actions Bot added ASR NLP Multi Modal Speaker Tasks TTS Audio core common CI labels Aug 31, 2026

Copilot AI reviewed Aug 31, 2026

View reviewed changes

Jakob Drachmann Havtorn (JakobHavtorn) closed this Aug 31, 2026

Jakob Drachmann Havtorn (JakobHavtorn) mentioned this pull request Aug 31, 2026

feat: rebase fork onto upstream HEAD with all custom Corti commits #3

Open

Navigation Menu

chore: rebase fork to NVIDIA/NeMo upstream HEAD - #2

chore: rebase fork to NVIDIA/NeMo upstream HEAD#2
Jakob Drachmann Havtorn (JakobHavtorn) wants to merge 397 commits into
mainfrom
chore/rebase-upstream-sync

Jakob Drachmann Havtorn (JakobHavtorn) commented Aug 31, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Jakob Drachmann Havtorn (JakobHavtorn) commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

Jakob Drachmann Havtorn (JakobHavtorn) commented Aug 31, 2026

Summary

What's included

Why

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Jakob Drachmann Havtorn (JakobHavtorn) commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants