Skip to content

fix: replace linear OpenSSF Scorecard map with banded mapping (IN-1247) - #4520

Merged
joanagmaia merged 1 commit into
chore/health-score-rollup-in-1246from
chore/health-score-openssf-in-1247
Aug 28, 2026
Merged

fix: replace linear OpenSSF Scorecard map with banded mapping (IN-1247)#4520
joanagmaia merged 1 commit into
chore/health-score-rollup-in-1246from
chore/health-score-openssf-in-1247

Conversation

@joanagmaia

Copy link
Copy Markdown
Contributor

Summary

  • Replaces round(min(raw, 10) × 0.7) with a banded mapping calibrated to the real raw score distribution (median ≈ 4, p99 ≈ 7.9)
  • Bands: raw ≥ 7 → 7 pts, ≥ 5.5 → 5, ≥ 4 → 4, ≥ 2.5 → 2, else 0
  • 7-point weight and scorecardAvailable gate unchanged; no schema changes downstream

Effect: repos reaching full Scorecard points 41 → ~889; mean security score among scored repos 21.1 → 21.5. Project-level impact grows as scan coverage (~25% of LF repos) improves.

Out of scope: legacy ossPackages_enriched.pipe v1 linear mapping — left for v1 deprecation; public-API × 10 scale mappings in openapi.yaml files (different scale, unrelated).

Post-merge: replay health_score_v2_security_ds, health_score_v2_signal_detail_ds, and project_insights_health_breakdown_ds.

Closes IN-1247

Copilot AI balanced review requested due to automatic review settings August 27, 2026 18:06
@cursor

cursor Bot commented Aug 27, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes scoring logic for a core health metric consumed by multiple Tinybird datasources; behavior shifts broadly across repos but gates and schema stay the same, with replay needed for consistency.

Overview
Replaces the linear OpenSSF Scorecard → points conversion in health_score_v2_security.pipe with a banded multiIf mapping so full 7-point credit aligns with realistic raw scores (median ~4, p99 ~7.9) instead of requiring raw ≥ 9.3 under round(min(raw,10)×0.7).

Bands: raw ≥ 7 → 7 pts, ≥ 5.5 → 5, ≥ 4 → 4, ≥ 2.5 → 2, else 0. The 7-point weight, scorecardAvailable gate, and downstream column names are unchanged; only how scorecardScorePts is computed changes, which flows into the security category’s graceful-degradation math.

Pipe description documents the calibration rationale (IN-1247). Expect more repos at max Scorecard contribution and slightly higher mean security scores until related datasources are replayed post-merge.

Reviewed by Cursor Bugbot for commit 5f3bc43. Bugbot is set up for automated code reviews on this repo. Configure here.

@joanagmaia
joanagmaia force-pushed the chore/health-score-openssf-in-1247 branch from 867ac00 to ebb5ce5 Compare August 27, 2026 18:08
@joanagmaia
joanagmaia changed the base branch from main to chore/health-score-rollup-in-1246 August 27, 2026 18:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Recalibrates OpenSSF Scorecard scoring to reflect observed score distributions.

Changes:

  • Replaces linear scoring with threshold bands.
  • Documents the new mapping and rationale.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread services/libs/tinybird/pipes/health_score_v2_security.pipe Outdated
Copilot AI review requested due to automatic review settings August 27, 2026 18:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (1)

services/libs/tinybird/pipes/health_score_v2_security.pipe:95

  • This changes the scoring methodology, but this pipe and the final health_score_v2.pipe output still label results as methodologyVersion = '2.0.0'. After the planned replay, scores produced by the old and new mappings are indistinguishable, while raw snapshots are also keyed by that version. Please bump the methodology version consistently across the category, aggregate/signal-detail, and raw-snapshot producers (for example, to 2.0.1).
                            multiIf(

Copilot AI review requested due to automatic review settings August 27, 2026 18:12
@joanagmaia
joanagmaia force-pushed the chore/health-score-openssf-in-1247 branch from ebb5ce5 to 5e16494 Compare August 27, 2026 18:12
@joanagmaia
joanagmaia force-pushed the chore/health-score-openssf-in-1247 branch from 5e16494 to b007848 Compare August 27, 2026 18:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings August 27, 2026 18:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@joanagmaia
joanagmaia force-pushed the chore/health-score-openssf-in-1247 branch from b007848 to f17e0fa Compare August 27, 2026 18:18
Copilot AI review requested due to automatic review settings August 27, 2026 18:18
@joanagmaia
joanagmaia requested a review from gaspergrom August 27, 2026 18:19
@joanagmaia joanagmaia self-assigned this Aug 27, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@joanagmaia
joanagmaia force-pushed the chore/health-score-openssf-in-1247 branch from f17e0fa to d4ee3e4 Compare August 27, 2026 18:20
Copilot AI review requested due to automatic review settings August 27, 2026 18:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

gaspergrom
gaspergrom previously approved these changes Aug 28, 2026
Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>
Copilot AI review requested due to automatic review settings August 28, 2026 08:54
@joanagmaia
joanagmaia force-pushed the chore/health-score-openssf-in-1247 branch from d4ee3e4 to 5f3bc43 Compare August 28, 2026 08:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@joanagmaia
joanagmaia merged commit 37cdc18 into main Aug 28, 2026
22 of 23 checks passed
@joanagmaia
joanagmaia deleted the chore/health-score-openssf-in-1247 branch August 28, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants