Skip to content

Fixed flaky test_optional by using seed and larger sample size.#2376

Merged
fcurella merged 2 commits into
joke2k:masterfrom
tejasae-afk:fix/flaky-test-optional-range
Jun 8, 2026
Merged

Fixed flaky test_optional by using seed and larger sample size.#2376
fcurella merged 2 commits into
joke2k:masterfrom
tejasae-afk:fix/flaky-test-optional-range

Conversation

@tejasae-afk

Copy link
Copy Markdown
Contributor

Fixes #2366.

test_optional drew only 10 samples from a probabilistic generator and asserted that all three outcomes (True, False, None) appeared. With the default probability split, the empirical failure rate was ~12% per run (confirmed by 5000-run Monte Carlo in the issue report).

Fix:

  • Seed the generator with Faker.seed(0) for determinism.
  • Increase the sample size from 10 to 100. At worst-case probability splits the chance of missing any outcome in 100 draws is astronomically small (< 0.000003%).

No behaviour change — only the test is affected.

@dancergraham

Copy link
Copy Markdown
Contributor

linting is now fixed - if you make a merge commit or similar minor update then the ci should run normally

Seeding made the test deterministic against a fixed sequence rather than
verifying probabilistic coverage. 100 draws reduce failure probability to
< 0.000003% without relying on a specific seed. Trims the 3-line comment
to a single reference line.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@tejasae-afk

Copy link
Copy Markdown
Contributor Author

Done! thanks.

@fcurella fcurella left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@fcurella fcurella merged commit dc63614 into joke2k:master Jun 8, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky test: tests/test_optional.py::test_optional relies on probabilistic assertion

3 participants