Skip to content

feat: support diarized Corti transcript input with speaker-aware analysis - #2

Open
Dan Engel (dwe2501) wants to merge 10 commits into
mainfrom
phase-1-support-diarized-transcript-input-with-speaker-aware-sr-3009
Open

Dan Engel (dwe2501) wants to merge 10 commits into
mainfrom
phase-1-support-diarized-transcript-input-with-speaker-aware-sr-3009

Conversation

@dwe2501

@dwe2501 Dan Engel (dwe2501) commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds end-to-end support for diarized Corti transcripts: import, speaker-aware display, per-speaker evaluation, diarization error detection, and a diarization accuracy metric.

What changed

Phase 1 — Speaker-aware transcript import & display

  • New tympany/diarize.py parser auto-detects Corti WebSocket streams and REST /transcripts JSON formats, extracts speakerId/channel, and builds per-word speaker labels
  • "Import Corti transcript" input mode on the Create page (paste or upload JSON)
  • Speaker labels propagated through Token → DiffGroup → Sample → error rows on the results page
  • Speaker column, speaker filter, and speaker-tagged CSV export on results page
  • BeWER report view shows speaker labels per alignment example

Phase 2 — Per-speaker evaluation

  • run_bewer_diarized() in bewer_eval.py splits the transcript by speaker and runs bewer separately per speaker
  • "Evaluate per speaker" checkbox on the Create page (visible with Corti transcript input)
  • Per-speaker metrics table (WER, CER, updated WER/CER, ref word count) on results and BeWER report pages
  • Per-speaker re-run support (recomputes per-speaker metrics after excluding errors)
  • Alignment examples split by speaker in the BeWER report view via _split_ops_by_speaker()

Phase 3 — Diarization error analysis

New tympany/diarize_errors.py with align_segments() detecting 5 error types:

  • Speaker mismatch — same time range, different speaker (medium risk)
  • Merged turns — multiple ref speakers → one gen segment (medium risk)
  • Split turns — one ref segment → multiple gen speakers (medium risk)
  • Missing turn — ref segment with no gen counterpart (high risk)
  • Extra turn — gen segment with no ref counterpart (high risk)
    New diarization_error classification in categorize.py with its own color, dropdown option, breakdown column, and filter on the results page
    diarization_accuracy() metric (matched turns / total turns, coverage ≥ 50% threshold) displayed on results and BeWER report pages

README — updated with new sections for diarized transcript support, per-speaker evaluation, diarization error analysis, updated error classification table, project structure, and development notes.

Test coverage

  • 150 tests pass (12 new test functions across test_diarize.py, test_diarize_errors.py, test_bewer.py, and test_report_render.py)
  • Test fixtures include clean diarized transcripts and transcripts with deliberate diarization errors
  • All new fields default to empty/None/() for backward compatibility with existing non-diarized analyses

@linear-code

linear-code Bot commented Sep 2, 2026

Copy link
Copy Markdown

SR-3009

@dwe2501 Dan Engel (dwe2501) changed the title support diarized transcript input with speaker aware display in output feat: support diarized Corti transcript input with speaker-aware analysis Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant