Skip to content

Enhance triage skills with updated scoring, confidence, and P-scale labels#6294

Open
lauren-ciha wants to merge 20 commits intomainfrom
user/laurenciha/weekly-sync-scoring
Open

Enhance triage skills with updated scoring, confidence, and P-scale labels#6294
lauren-ciha wants to merge 20 commits intomainfrom
user/laurenciha/weekly-sync-scoring

Conversation

@lauren-ciha
Copy link
Copy Markdown
Member

@lauren-ciha lauren-ciha commented Mar 12, 2026

This PR builds upon the issue-triage and feature-area-report skills.

The key changes are:

  • Added a Validate-FeatureAreaReport.ps1 that checks the report against the live GitHub label data using gh cli
  • Updated contacts.json to include a list of contacts rather than a primary/secondary contact
  • Updated scoring for issue priority to give further weight to community feedback
  • Updated "P-rating" priority rankings
  • Added confidence scoring to each part of the skill

Copy link
Copy Markdown
Contributor

@guimafelipe guimafelipe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this is a script that we use internally, I don't think any of my comments should be blocking. But maybe they will help making this overall process more maintainable and consistent.

Feel free to take them as suggestions.

@lauren-ciha lauren-ciha force-pushed the user/laurenciha/weekly-sync-scoring branch 2 times, most recently from 1fcc9b3 to c0833b4 Compare March 25, 2026 16:41
…abels

Scoring System:
- New weights: reactions=30%, age=30%, comments=30%, severity=10%
- P-scale severity labels: P0=critical, P1=high, P2=medium, P3=low
- Confidence scoring with grep-friendly [confidence:XX] format (0-100)

Label Consolidation:
- Merged Hot + Popular into Popular (>=5 reactions threshold)

Contact Schema:
- Simplified from {primary, secondary} to single {contact} field
- Removed legacy schema backward compatibility

Area Suggestions:
- Get-IssueDetails.ps1 dynamically fetches area labels via Get-RepositoryLabels.ps1
- area-Notifications covers all notification types (toast, badge, push, wns)
- Fixed area-PowerManagement naming

Documentation:
- Added PowerShell examples alongside Bash for confidence filtering
- Updated all SKILL.md files with new configuration details
@lauren-ciha lauren-ciha force-pushed the user/laurenciha/weekly-sync-scoring branch from 25e581c to 763cff9 Compare March 25, 2026 20:00
- Add RawUpdateAgeDays to Get-IssueScore (computed from updatedAt)
- Require comments >= trending_comments AND updated within trending_days
  in both Get-HighlightLabels (ReportLib) and Get-DetailedIssueScore
- Fix scoring-algorithm.md: comments >= 5 -> >= 10, add trending_days
  to highlight table and thresholds reference
- Add missing trending_days to SKILL.md config example
| 3-5 | 6 |
| 6-10 | 10 |
| 11+ | 15 |
The same 20/25/40/50/67/75/80/100% bracket pattern applies to `age` and `comments`. Changing the weight value scales all brackets proportionally.
Copy link
Copy Markdown
Member

@Scottj1s Scottj1s Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what bracket pattern is this referring to? doesn't appear to align with the reactions table. I think maintaining explicit tables for age, comments is best.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason this isn't part of ReportLib.ps1?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants