Skip to content

FixContinuationStyle: sanitize each paragraph once instead of twice - #14017

Merged
niksedk merged 1 commit into
mainfrom
claude/fix-continuation-style-sanitize-once
Aug 23, 2026
Merged

FixContinuationStyle: sanitize each paragraph once instead of twice#14017
niksedk merged 1 commit into
mainfrom
claude/fix-continuation-style-sanitize-once

Conversation

@niksedk

@niksedk niksedk commented Aug 23, 2026

Copy link
Copy Markdown
Member

Summary

FixContinuationStyle.Fix walks paragraph pairs and called ContinuationUtilities.SanitizeString on both p and pNext every iteration — so every paragraph was sanitized twice: once as pNext's text, then again as p's text one iteration later. SanitizeString runs four regex replaces per call, and it is the single largest cost in this rule.

This carries the sanitized "next" value forward instead. Two details make it exact:

  • the carry is captured before the Arabic conversion, which the next iteration reapplies;
  • the carry is dropped whenever the loop writes pNext.Text, so the following iteration re-sanitizes what the paragraph actually holds.

Benchmark

Apple M4, .NET 10.0.7, Release, best-of-3 with a 300-pass warmup, new Subtitle(...) copy cost subtracted.

corpus before after
general 1500-cue file 0.926 ms 0.654 ms −29%
continuation-heavy 1200-cue file 2.274 ms 2.111 ms −7%

SanitizeString calls over the 1500-cue file: 3462 → 1964 (−43%).

The continuation-heavy corpus gains less precisely because more paragraphs get rewritten, which drops the carry — the safety path doing its job.

Correctness

  • Output-identical across 192 configurations: 2 corpora × 12 continuation styles × 4 languages × 2 lyrics settings, comparing every resulting paragraph text and every callback log entry (AllowFix / AddFixToListView / UpdateFixStatus). 85,116 recorded fix events, identical checksums.
  • 2.4M randomized rule invocations comparing carry against no-carry: no divergence.
  • LibSETests: 1425/1425 pass. LibSE.csproj builds clean for both netstandard2.1 and net10.0.

Note: the carriedText = null invalidation never actually changed output in those 2.4M combinations — the predicates consuming the sanitized text are prefix-insensitive by design. It is kept so the equivalence is structural rather than dependent on that property continuing to hold.

Tests

  • FixContinuationStyleTest — pins a case that genuinely discriminates a correct carry from a wrong one. Worth noting: a plain continuation chain does not — it passes even with a deliberately broken carry, so that version was discarded in favour of one found by brute-force search. Mutation-checked: swapping textNext for text in the carry makes it fail.
  • UtilitiesCountTagInTextTestUtilities.CountTagInText had no direct tests, despite the char overload being about to have two implementations to keep in step (Utilities: vectorize CountTagInText(string, char) on .NET 8+ #13991). Covers empty, no-match, adjacent hits, and a hit on the final index (where the netstandard2.1 loop returns early from inside its body), plus agreement between the char and string overloads. Deliberately does not pin null behaviour, which differs by target framework.

🤖 Generated with Claude Code

The rule walks paragraph pairs and called ContinuationUtilities.SanitizeString
on both p and pNext every iteration, so every paragraph was sanitized twice:
once as pNext's text, then again as p's text one iteration later. SanitizeString
runs four regex replaces per call.

Carry the sanitized "next" value forward instead. The carry is captured before
the Arabic conversion (which the next iteration reapplies) and dropped whenever
the loop writes pNext.Text, so the following iteration re-sanitizes what the
paragraph actually holds.

Over a 1500-cue file: SanitizeString calls 3462 -> 1964 (-43%), and the rule
goes 0.926 ms -> 0.654 ms (-29%). On a continuation-heavy 1200-cue file, where
more paragraphs get rewritten and drop the carry, 2.274 ms -> 2.111 ms (-7%).

Verified output-identical across 192 configurations (2 corpora x 12 continuation
styles x 4 languages x 2 lyrics settings), comparing every paragraph text and
every callback log entry - 85,116 recorded fix events, identical checksums. A
further 2.4M randomized rule invocations comparing carry against no-carry found
no divergence.

Adds tests: FixContinuationStyleTest pins a case that actually discriminates a
correct carry from a wrong one (a plain continuation chain does not - it passes
either way), and UtilitiesCountTagInTextTest covers Utilities.CountTagInText,
which had no direct tests despite having two implementations to keep in step.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@niksedk
niksedk merged commit aadb703 into main Aug 23, 2026
1 check passed
@niksedk
niksedk deleted the claude/fix-continuation-style-sanitize-once branch August 23, 2026 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant