Repository navigation
np.nan in mne.stats.permutation_t_test fails silently #13767
Description
Activity
- changed the title
[-]np.nan in mne.stats.permutation_t_test[/-][+]np.nan in mne.stats.permutation_t_test fails silently[/+]on Mar 18, 2026 Yeah we probably want a
np.isfinite(x).all()check in there. Probably lots of functions in MNE could benefit from this but to keep scope and review manageable but still extensible I think adef _validate_data(data):inmne/utilssomewhere thatmne/statsuses could make sense. (I like using a private function here in case we eventually want to add other validations like dim or shape or whatever we can with newkwargs, and for now just have it returnnp.isfinite(data).all()...)Do you want me to do a PR?
Sure that would be great!
Yeah we probably want a
np.isfinite(x).all()check in there. Probably lots of functions in MNE could benefit from this but to keep scope and review manageable but still extensible I think adef _validate_data(data):inmne/utilssomewhere thatmne/statsuses could make sense. (I like using a private function here in case we eventually want to add other validations like dim or shape or whatever we can with newkwargs, and for now just have it returnnp.isfinite(data).all()...)actually this function already exists! However it raises a
ValueErrorinstead of printing aRuntimeWarningdef _check_if_nan(data, msg=" to be plotted"): """Raise if any of the values are NaN.""" if not np.isfinite(data).all(): raise ValueError(f"Some of the values {msg} are NaN.")
Should I extend that function or create a new
_validate_datathat calls that and acts as a basis for future checks?Should I extend that function or create a new
_validate_datathat calls that and acts as a basis for future checks?extend it with an
on_nanoron_presentor similarly-named param, which takes values "error", "warn" (usually we do error|warn|ignore, but I don't see the point of "ignore" in this case)default should be "error" so you avoid needing to pass the param in all the existing places where it's already in use
The silent failure stems from how NumPy handles NaN propagation: arithmetic on NaN values (e.g.,
NaN + x,NaN * x) silently produces NaN without triggering anyRuntimeWarning, whereas the zero-column case happens to produce a0/0division at line 20 ofmne/stats/permutations.py(np.abs(mus) / (stds / sqrt(n_samples))), which does trip numpy's invalid-value warning. Since there's no explicit NaN guard at the entry point ofpermutation_t_test(or the other stats functions), the NaN propagates all the way through the permutation loop and intoh0, ultimately making every p-value 0 or 1 depending on how the comparison against the NaN distribution resolves — a completely misleading result. Adding anp.isnan(X).any()check with an explicitValueErroror at minimum awarnings.warn(... RuntimeWarning)at the top of the affected functions (likely a shared utility so it covers the whole module) would catch this before any computation begins.
I've recently encountered a situation where I had a
np.nanin the data I was running apermutation_t_teston. This failed (expectedly), however, it did not give me any warning/error! I only realized because apvalue=0seemed a bit suspicious in my plotI think that it should at least print a warning, similar to how it give a
RuntimeWarningwhen one test's observations are all zero (probably due to underlying numpy warnings.)I assume this is the same for other functions in the stats module. If this is something that should be improved, probably best to do it systematically for all functions?