Skip to content

fix: cast y to float64 for numba crand in Moran_Local and G_Local to avoid TypingError.#427

Open
samay2504 wants to merge 2 commits intopysal:mainfrom
samay2504:fix/issue-376-moran-local-float32-dtype
Open

fix: cast y to float64 for numba crand in Moran_Local and G_Local to avoid TypingError.#427
samay2504 wants to merge 2 commits intopysal:mainfrom
samay2504:fix/issue-376-moran-local-float32-dtype

Conversation

@samay2504
Copy link
Copy Markdown
Contributor

fixes #376.

This fixes issue where passing a float32 array to Moran_Local causes a numba TypingError. When you pass a float32 array to Moran_Local, Moran_Local_BV or G_Local the internal standardised array keeps that float32 type. The conditional randomisation engine then casts the spatial weights to match this type and that creates a nasty mismatch inside the numba compiled functions when they try to multiply float32 and float64 values together. I fixed this by casting the input arrays to float64 right at the start of these functions to make sure the internal arrays always have the right type. It completely gets rid of the numba error and lets conditional randomisation work perfectly for float32 inputs without changing the behaviour for standard float64 arrays. I also threw in some regression tests for all three methods so we don't see this problem creeping back in.

Copilot AI review requested due to automatic review settings March 16, 2026 10:08
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses a numba TypingError triggered when float32 input arrays are passed into conditional randomization paths for local statistics by ensuring inputs are cast to float64 early.

Changes:

  • Cast y to np.float64 at the start of Moran_Local.
  • Cast x/y to np.float64 at the start of Moran_Local_BV.
  • Cast y to np.float64 at the start of G_Local to stabilize the crand/numba dot-product dtype behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
esda/moran.py Forces Moran_Local and Moran_Local_BV inputs to float64 to prevent dtype mismatches in numba-compiled conditional randomization.
esda/getisord.py Forces G_Local inputs to float64 to prevent dtype mismatches in numba-compiled conditional randomization.
Comments suppressed due to low confidence (2)

esda/moran.py:1783

  • Avoid using assert for runtime input validation here (asserts are stripped with python -O). Please raise a ValueError (or TypeError) when x and y lengths differ so this check is enforced in production.
        self.y = y
        self.x = x
        n = len(y)
        assert len(y) == len(x), "x and y must have the same shape!"
        self.n = n

esda/moran.py:1309

  • Regression coverage appears to be missing for the float32/Numba conditional-randomization path. Please add a test that Moran_Local with y.astype(np.float32) and permutations>0 runs without raising a numba.core.errors.TypingError (skip if numba unavailable) and produces finite p_sim/sim outputs.
        y = np.asarray(y, dtype=np.float64).flatten()
        self.y = y
        n = len(y)
        self.n = n
        self.n_1 = n - 1

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread esda/moran.py Outdated
Comment on lines 1777 to 1779
x = np.asarray(x, dtype=np.float64).flatten()
y = np.asarray(y, dtype=np.float64).flatten()
self.y = y
Comment thread esda/getisord.py Outdated
Comment on lines +598 to +600
assert transform.lower() in ("r", "b"), (
f'Transforms must be binary "b" or row-standardized "r".Recieved: {transform}'
)
Comment thread esda/getisord.py Outdated
def _infer_star_and_structure_w(weights, star, transform):
assert transform.lower() in ("r", "b"), (
f'Transforms must be binary "b" or row-standardized "r".Recieved: {transform}'
Comment thread esda/getisord.py Outdated
Comment on lines +418 to +422
y = np.asarray(y, dtype=np.float64).flatten()
self.n = len(y)
self.y = y
w, star = _infer_star_and_structure_w(w, star, transform)
if isinstance(w, W):
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 16, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.3%. Comparing base (59d633c) to head (01704e9).
⚠️ Report is 58 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##            main    #427     +/-   ##
=======================================
- Coverage   82.7%   82.3%   -0.4%     
=======================================
  Files         27      27             
  Lines       3833    3993    +160     
=======================================
+ Hits        3170    3288    +118     
- Misses       663     705     +42     
Files with missing lines Coverage Δ
esda/getisord.py 71.9% <100.0%> (+1.3%) ⬆️
esda/moran.py 85.2% <100.0%> (+0.3%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@martinfleis
Copy link
Copy Markdown
Member

Fix the diff please. This doesn't require a change of 728 lines.

…d replace assert with ValueError in getisord.py
@samay2504 samay2504 force-pushed the fix/issue-376-moran-local-float32-dtype branch from 516bdf9 to e21c8ab Compare March 16, 2026 14:30
@samay2504
Copy link
Copy Markdown
Contributor Author

Thanks for the feedback @martinfleis , the issue has been addressed.

@ljwolf
Copy link
Copy Markdown
Member

ljwolf commented Mar 16, 2026

Hi! thank you for this contribution. Casting all input to float64 is not the way that #376 should be addressed. Instead, we should define the type signature in numba to accept valid floats at any size.

@samay2504
Copy link
Copy Markdown
Contributor Author

Thanks for the guidance @ljwolf , I’ve updated this to use explicit Numba eager signatures for both float32 and float64 paths instead of coercing inputs to float64. I removed the blanket casts in esda/moran.py and esda/getisord.py, added typed local crand signatures for univariate and bivariate kernels, and added targeted float32 regression coverage in esda/tests/test_moran.py and esda/tests/test_getisord.py. The focused float32 tests are passing on this branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Moran_Local fails when numba installed – Spatial Autocorrelation notebook

4 participants