Skip to content

fix: prevent duplicate PR comments via similarity-based deduplication (issue #1495)#1496

Open
konard wants to merge 5 commits into
mainfrom
issue-1495-2031b20bafff
Open

fix: prevent duplicate PR comments via similarity-based deduplication (issue #1495)#1496
konard wants to merge 5 commits into
mainfrom
issue-1495-2031b20bafff

Conversation

@konard

@konard konard commented Mar 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #1495 — Multiple repeated comments were posted after ready to merge on link-foundation/sandbox#73. After an AI work session completed, 4+ duplicate "Validation Complete" comments were posted within a 37-second window.

Root Cause

The hive-mind system had no deduplication mechanism for:

  1. AI agent-generated validation/summary comments
  2. Cross-session duplicate comments (different processes posting to the same PR)
  3. Post-session comment leaks from concurrent processes

Changes

  • New comment-dedup.lib.mjs: Text normalization, overlap-coefficient similarity scoring, and deduplication check before posting comments
    • normalizeCommentForComparison() — strips markdown, timestamps, emoji, table formatting
    • computeSimilarity() — overlap coefficient on keyword sets
    • computeCommentSimilarity() — combines header matching with word overlap for better accuracy
    • checkForRecentSimilarComment() — checks recent PR comments within a time window
    • postCommentWithDedup() — drop-in replacement for gh pr comment with built-in dedup
  • Integrated dedup into solve.auto-merge.lib.mjs: "Ready to merge" and "Auto-merged" comments now use postCommentWithDedup() with 10-minute window and 0.7 similarity threshold
  • AI agent prompt enhancement: Added "Comment deduplication" section to all 4 prompt templates (claude, codex, opencode, agent) instructing the AI to check for existing similar comments before posting
  • Case study documentation: Timeline reconstruction and root cause analysis in docs/case-studies/issue-1495/
  • Changeset: Patch version bump

Test Plan

Adding .gitkeep for PR creation (default mode).
This file will be removed when the task is complete.

Issue: #1495
@konard konard self-assigned this Mar 29, 2026
Issue #1495: Multiple repeated comments were posted after `ready to merge`
on link-foundation/box#73. Root cause: no deduplication for AI-generated
or system-generated PR comments across sessions.

Changes:
- Add comment-dedup.lib.mjs: text normalization, overlap-coefficient
  similarity scoring, and deduplication check before posting
- Integrate postCommentWithDedup() into auto-merge "Ready to merge"
  and "Auto-merged" comment posting paths
- Add "Comment deduplication" section to all AI agent system prompts
  (claude, codex, opencode, agent) instructing the AI to check for
  existing similar comments before posting
- Add case study documentation with timeline reconstruction
- Add 26 unit tests for normalization, similarity, and real-world
  duplicate detection

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@konard konard changed the title [WIP] Multiple repeated comments after ready to merge without clear demarcation of working sessions fix: prevent duplicate PR comments via similarity-based deduplication (issue #1495) Mar 29, 2026
The no-misleading-character-class rule flagged the combined character
ranges. Use \p{Emoji_Presentation} instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@konard
konard marked this pull request as ready for review March 29, 2026 18:29
@konard

konard commented Mar 29, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Public pricing estimate: $15.698279
  • Calculated by Anthropic: $10.339753 USD
  • Difference: $-5.358526 (-34.13%)

🤖 Models used:

  • Tool: Anthropic Claude Code
  • Requested: opus
  • Main model: Claude Opus 4.6 (claude-opus-4-6)
  • Additional models:
    • Claude Haiku 4.5 (claude-haiku-4-5-20251001)

📎 Log file uploaded as Gist (3579KB)


Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard

konard commented Mar 29, 2026

Copy link
Copy Markdown
Contributor Author

🔄 Auto-restart 1/3

Detected uncommitted changes from previous run. Starting new session to review and commit or discard them.

Uncommitted files:

?? gist-log.txt
?? pr73-comments.txt
?? pr73-issue-comments.json

Auto-restart will stop after changes are committed or discarded, or after 2 more iterations. Please wait until working session will end and give your feedback.

Download session log (23K lines) and PR #73 comments JSON
to docs/case-studies/issue-1495/ for root cause analysis
of duplicate comment incident.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@konard

konard commented Mar 29, 2026

Copy link
Copy Markdown
Contributor Author

🔄 Auto-restart 1/3 Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Public pricing estimate: $3.911514
  • Calculated by Anthropic: $2.240721 USD
  • Difference: $-1.670793 (-42.71%)

🤖 Models used:

  • Tool: Anthropic Claude Code
  • Requested: opus
  • Model: Claude Opus 4.6 (claude-opus-4-6)

📎 Log file uploaded as Gist (4547KB)


Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard

konard commented Mar 29, 2026

Copy link
Copy Markdown
Contributor Author

✅ Ready to merge

This pull request is now ready to be merged:

  • All CI checks have passed
  • No merge conflicts
  • No pending changes

Monitored by hive-mind with --auto-restart-until-mergeable flag

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multiple repeated comments after ready to merge without clear demarcation of working sessions

1 participant