[Stack 11/17] Fix D9: z-score thresholds from two-tailed to one-tailed#2518
Open
jucor wants to merge 1 commit intospr/edge/b9062b50from
Open
[Stack 11/17] Fix D9: z-score thresholds from two-tailed to one-tailed#2518jucor wants to merge 1 commit intospr/edge/b9062b50from
jucor wants to merge 1 commit intospr/edge/b9062b50from
Conversation
This was referenced Mar 30, 2026
Open
Open
jucor
changed the title
jucor
force-pushed
the
spr/edge/0194003d
branch
2 times, most recently
from
1452e78 to
24de40d
Compare
## Summary - Fix D9: change z-score significance thresholds from two-tailed to one-tailed, matching Clojure's `stats.clj` - `Z_90`: 1.645 → 1.2816, `Z_95`: 1.96 → 1.6449 - Also resolves an internal inconsistency — Python's own `stats.py` already used the correct one-tailed values ## Why one-tailed? The proportion tests in Polis check whether a comment's agree (or disagree) rate is **significantly above 0.5** — a directional hypothesis. One-tailed is correct because we only care about one direction at a time. The two-tailed values were 28% more conservative, causing fewer comments to pass significance. ## Test plan - [x] TDD: removed xfail from 3 D9 tests, confirmed red (3 failures), applied fix, confirmed green - [x] Discrepancy tests: 63 passed, 6 skipped, 50 xfailed (all 7 datasets including private) - [x] Regression tests: 19 passed (all 7 datasets, golden snapshots re-recorded) - [x] Repness unit tests: 36 passed (boundary values updated to match new thresholds) - [x] 4 pre-existing failures unrelated to D9 (PCA incremental blobs, DB-dependent tests) 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Squashed commits - Plan: add task parallelization analysis for remaining fixes - Fix D9: match Clojure z-sig semantics (strict >, no abs) and remove dead stats.py - Re-record vw golden snapshot after D9 z-sig semantics change - Update plan: mark D9 as done, note stats.py removal for next PR - Add mathematical rigor and exhaustive testing guidance to fix plan - Plan: move PR 14 earlier (prerequisite for blob tests) + add handoff doc - Re-record golden snapshots after upstream cascade commit-id:0194003d
jucor
force-pushed
the
spr/edge/0194003d
branch
from
24de40d to
add1343
Compare
Delphi Coverage Report
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
stats.cljZ_90: 1.645 → 1.2816,Z_95: 1.96 → 1.6449stats.pyalready used the correct one-tailed valuesWhy one-tailed?
The proportion tests in Polis check whether a comment's agree (or disagree) rate is significantly above 0.5 — a directional hypothesis. One-tailed is correct because we only care about one direction at a time. The two-tailed values were 28% more conservative, causing fewer comments to pass significance.
Test plan
🤖 Generated with Claude Code
Squashed commits
commit-id:0194003d
Stack: