Skip to content

Fixing ESS overvote discrepancy bug #2100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
umbernhard wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fixing ESS overvote discrepancy bug #2100

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions server/api/shared.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -527,15 +527,24 @@ def ballot_vote_deltas(
reported = {choice.id: "0" for choice in contest.choices}

deltas = {}
total_audited_votes = 0
for choice in contest.choices:
reported_vote = (
0 if reported[choice.id] in ["o", "u"] else int(reported[choice.id])
)
audited_vote = (
0 if audited[choice.id] in ["o", "u"] else int(audited[choice.id])
)
total_audited_votes += audited_vote
deltas[choice.id] = reported_vote - audited_vote

# If this is an overvote, and that was expected, there's no discrepancy here.
# TODO: do we need to do this for undervotes too?
if contest.votes_allowed and total_audited_votes > contest.votes_allowed:
for choice in contest.choices:
if reported[choice.id] == "o":
deltas[choice.id] = 0

if all(delta == 0 for delta in deltas.values()):
return None

Expand Down
2 changes: 1 addition & 1 deletion server/tests/ballot_comparison/snapshots/snap_test_ballot_comparison.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@
J1,0002,BATCH2,5,0002000175,"Round 1: 0.128937575131137250, 0.240487859312182291",AUDITED,Choice 1-2,Choice 1-2,,,Choice 2-2,Choice 2-2,,\r
J2,0001,BATCH1,1,0001013415,Round 1: 0.228946820159681463,AUDITED,"Choice 1-1, INVALID_WRITE_IN",Choice 1-1,,,"Choice 2-1, INVALID_WRITE_IN",Choice 2-1,,\r
J2,0001,BATCH1,3,0001013417,Round 1: 0.457121710197159606,AUDITED,Choice 1-1,Choice 1-1,,,Choice 2-1,Choice 2-1,,\r
J2,0001,BATCH2,3,0001000417,Round 1: 0.269793733438455805,AUDITED,"Choice 1-1, Choice 1-2",Overvote,Choice 1-1: -1; Choice 1-2: -1,,"Choice 2-1, Choice 2-3",Overvote,Choice 2-1: -1; Choice 2-3: -1,\r
J2,0001,BATCH2,3,0001000417,Round 1: 0.269793733438455805,AUDITED,"Choice 1-1, Choice 1-2",Overvote,Choice 1-2: -1,,"Choice 2-1, Choice 2-3",Overvote,,\r

@arsalansufi arsalansufi Aug 20, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm looking at this snapshot diff more closely and noticing something off.

The following

Audit result: Choice 1-1, Choice 1-2
CVR result:   Overvote

is still resulting in a discrepancy of Choice 1-2: -1 (previously Choice 1-1: -1; Choice 1-2: -1).

In contrast, the following

Audit result: Choice 2-1, Choice 2-3
CVR result:   Overvote

is expectedly resulting in no discrepancy (prevoiusly Choice 2-1: -1; Choice 2-3: -1).

I have a rough idea here what's going on and am prepping an edit to account for this. Because I can't push to this branch from a fork, I'll open a new PR.

@arsalansufi arsalansufi Aug 20, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the core issue is that, in this implementation, we only clear the discrepancy for a candidate if there's an "o" listed for that candidate in the reported object. But as far as I can tell, when a contest is overvoted, we may not have an "o" for every candidate. So rather than checking and clearing discrepancies on a per candidate basis, if we see any "o" listed, and the audit confirms an overvote, we should clear the discrepancy for all candidates.

Made that adjustment in #2161

@jonahkagan jonahkagan Aug 21, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, when a contest is overvoted, we do save an "o" for every candidate: https://github.com/votingworks/arlo/blob/main/server/api/cvrs.py#L885-L900

@arsalansufi arsalansufi Aug 21, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay good to know! Sound like there's an issue with the test setup then

J2,0002,BATCH1,3,0002003173,Round 1: 0.328294241227374952,AUDITED,Choice 1-2,Overvote,Choice 1-2: -1,1,Choice 2-2,Overvote,Choice 2-2: -1,1\r
J2,0002,BATCH2,1,0002000171,Round 1: 0.390715133294243377,AUDITED,Choice 1-1,Choice 1-1,,,Choice 2-3,Choice 2-3,,\r
J2,0002,BATCH2,2,0002000172,Round 1: 0.064290634474137509,AUDITED,Choice 1-1,Choice 1-1,,,Choice 2-3,Choice 2-3,,\r
Expand Down