diff --git a/services/libs/tinybird/datasources/project_insights_copy_ds.datasource b/services/libs/tinybird/datasources/project_insights_copy_ds.datasource index b8880956dc..bc1e9add00 100644 --- a/services/libs/tinybird/datasources/project_insights_copy_ds.datasource +++ b/services/libs/tinybird/datasources/project_insights_copy_ds.datasource @@ -32,7 +32,7 @@ DESCRIPTION > - `activeContributorsPrevious365Days` column is the unique count of active contributors in the previous 365 days (365-730 days ago). - `activeOrganizationsPrevious365Days` column is the unique count of active organizations in the previous 365 days (365-730 days ago). - `status` column is the current status of the project (e.g., 'active', 'archived'). - - `healthScoreV2` column is the Akrites-methodology composite health score (0-100). For project records (`type='project'`): median (`quantileExact(0.5)`) of per-repo scores across the project's enabled, non-excluded repos; null when no such repos exist or all have null scores. For repo records (`type='repo'`): per-repo score passed through from `health_score_v2_repo_copy_ds`. Distinct from the legacy `healthScore` column (community/contributor-based). + - `healthScoreV2` column is the Akrites-methodology composite health score (0-100). For project records (`type='project'`): sum of the three per-category medians (maintainerHealthScoreV2 + securitySupplyChainScoreV2 + developmentActivityScoreV2), null-guarded per IN-1248 (null when fewer than 2 categories are present). For repo records (`type='repo'`): per-repo score passed through from `health_score_v2_repo_copy_ds`. Distinct from the legacy `healthScore` column (community/contributor-based). - `healthLabel` column buckets `healthScoreV2`: 'excellent' (85+), 'healthy' (70-84), 'fair' (50-69), 'concerning' (30-49), 'critical' (<30). Lowercase, matches `ossPackages_enriched_ds.healthLabel` convention. - `lifecycleLabel` column is the project's aggregated maintenance state across its packages, using best-state-wins precedence (active > stable > declining > abandoned > archived) over `ossPackages_enriched_ds.lifecycleLabel`. - `impactScore` column is the average Osprey criticality `impact` (0-100) across the project's packages. Null when the project has no linked package data. diff --git a/services/libs/tinybird/pipes/project_insights.pipe b/services/libs/tinybird/pipes/project_insights.pipe index 1959192e0f..1762c5c5dd 100644 --- a/services/libs/tinybird/pipes/project_insights.pipe +++ b/services/libs/tinybird/pipes/project_insights.pipe @@ -13,7 +13,7 @@ DESCRIPTION > - `page`: Optional integer for pagination offset calculation, defaults to 0 - Response: Project records with all insights metrics including achievements as array of (leaderboardType, rank, totalCount) tuples - `healthScoreV2`, `healthLabel`, `lifecycleLabel`, `impactScore`, `impactLabel` are the Akrites-methodology fields (see `project_insights_copy_ds` for band definitions); may be null when the project has no linked package data. `healthScoreV2` is distinct from the legacy `healthScore` field. - - `maintainerHealthScoreV2` (0-40), `securitySupplyChainScoreV2` (0-35), `developmentActivityScoreV2` (0-25) are the project-level rollup of the Health Score v2 category breakdown (IN-1212) — independent medians (`quantileExact(0.5)`) over the same enabled, non-excluded repo set as `healthScoreV2`; they do not necessarily sum to `healthScoreV2`. Null when no repo in the project has a value for that category. + - `maintainerHealthScoreV2` (0-40), `securitySupplyChainScoreV2` (0-35), `developmentActivityScoreV2` (0-25) are the project-level rollup of the Health Score v2 category breakdown (IN-1212) — independent medians (`quantileExact(0.5)`) over the project's enabled, non-excluded repos. For project records, `healthScoreV2` is their sum (null-guarded per IN-1248). Null when no repo in the project has a value for that category. TAGS "Insights, Widget", "Project" diff --git a/services/libs/tinybird/pipes/project_insights_copy.pipe b/services/libs/tinybird/pipes/project_insights_copy.pipe index 1607a52a31..c472c67de1 100644 --- a/services/libs/tinybird/pipes/project_insights_copy.pipe +++ b/services/libs/tinybird/pipes/project_insights_copy.pipe @@ -90,7 +90,8 @@ DESCRIPTION > Lifecycle, Health Score v2, and Impact Score all rolled up to project level from the pre-computed per-repo values in health_score_v2_repo_copy_ds (materialized by health_score_v2.pipe — the expensive per-signal joins run there as their own copy job, not inline here). Per spec section 5/6: - - healthScoreV2: exact median (`quantileExact(0.5)`) across the project's enabled, non-excluded repos. + - healthScoreV2: sum of the three per-category medians (maintainer + security + development), null-guarded + per IN-1248 (NULL when fewer than 2 categories are present). See the node SQL for the full expression. - lifecycleLabelV2: best-state-wins (active > stable > declining > inert > abandoned > archived), with NULL (no usable signal, see health_score_v2_lifecycle.pipe) as the most-cautious outcome. `groupArray` silently drops NULL entries, so a project with at least one repo in a real state @@ -111,16 +112,27 @@ DESCRIPTION > - impactScoreRaw: MAX across the project's repos (each repo's own value is already a MAX over its published packages) — matches the spec's "MAX(packages.impact) over packages published by any repo in the project," since max-of-maxes = max-over-all. + - healthScoreV2: sum of the three per-category medians below, null-guarded per IN-1248 (NULL when + fewer than 2 categories are present). `quantileExactOrNull` is used instead of `quantileExact` so + the guard and coalesce can see NULL explicitly — `quantileExact` returns 0 on an empty set, which + would defeat the "< 2 categories → NULL" rule. - maintainerHealthScoreV2 / securitySupplyChainScoreV2 / developmentActivityScoreV2 (IN-1212): - independent per-category medians (`quantileExact(0.5)`) over repos where that category is non-NULL — - their repo set differs from healthScoreV2 (which includes repos with one imputed category), so they - do not necessarily reconcile with the project score. NULL when every repo is NULL for a category. + independent per-category medians (`quantileExact(0.5)`) over repos where that category is non-NULL. + NULL when every repo is NULL for a category. SQL > SELECT rep.insightsProjectId AS insightsProjectId, - quantileExact(0.5) - (hv2.healthScoreV2) AS healthScoreV2, + if( + (quantileExactOrNull(0.5)(hv2.maintainerHealthScoreV2) IS NOT NULL) + + (quantileExactOrNull(0.5)(hv2.securitySupplyChainScoreV2) IS NOT NULL) + + (quantileExactOrNull(0.5)(hv2.developmentActivityScoreV2) IS NOT NULL) + < 2, + NULL, + coalesce(quantileExactOrNull(0.5)(hv2.maintainerHealthScoreV2), 0) + + coalesce(quantileExactOrNull(0.5)(hv2.securitySupplyChainScoreV2), 0) + + coalesce(quantileExactOrNull(0.5)(hv2.developmentActivityScoreV2), 0) + ) AS healthScoreV2, if( empty(groupArray(hv2.lifecycleLabelV2)), NULL, diff --git a/services/libs/tinybird/pipes/project_repo_insights.pipe b/services/libs/tinybird/pipes/project_repo_insights.pipe index 1648cd8125..ec6637f603 100644 --- a/services/libs/tinybird/pipes/project_repo_insights.pipe +++ b/services/libs/tinybird/pipes/project_repo_insights.pipe @@ -12,7 +12,7 @@ DESCRIPTION > - `page`: Optional integer for pagination offset calculation, defaults to 0 - Response: Project and repository records with insights metrics - `healthScoreV2`, `healthLabel`, `lifecycleLabel`, `impactScore`, `impactLabel` are the Akrites-methodology fields (see `project_insights_copy_ds` for band definitions); may be null when the record has no linked package data. `healthScoreV2` is distinct from the legacy `healthScore` field. - - `maintainerHealthScoreV2` (0-40), `securitySupplyChainScoreV2` (0-35), `developmentActivityScoreV2` (0-25) are the Health Score v2 category breakdown (IN-1212). For project records: independent medians (`quantileExact(0.5)`) over the project's enabled, non-excluded repos — they do not necessarily sum to `healthScoreV2`. For repo records: per-repo values passed through from `health_score_v2_repo_copy_ds`; they sum to `healthScoreV2` only when all three categories are available; missing categories are imputed at the population median rate (IN-1249). Null when the record has no value for that category. + - `maintainerHealthScoreV2` (0-40), `securitySupplyChainScoreV2` (0-35), `developmentActivityScoreV2` (0-25) are the Health Score v2 category breakdown (IN-1212). For project records: independent medians (`quantileExact(0.5)`) over the project's enabled, non-excluded repos; `healthScoreV2` is their sum (null-guarded per IN-1248). For repo records: per-repo values passed through from `health_score_v2_repo_copy_ds`; they sum to `healthScoreV2` only when all three categories are available; missing categories are imputed at the population median rate (IN-1249). Null when the record has no value for that category. TAGS "Insights, Widget", "Project", "Repository"