Repository navigation
Claude code review #3875
Description
@MridulS asked Claude for a code review and here is the result.
We should check each point to see if it is valid and if so, fix it.
Scipp Codebase Deep Audit Report
Date: 2026-04-09
Scope: ~411 C++ files, ~91 Python source files, ~93 test files
Areas: Memory management, edge cases/UB, Python security, pybind11 bindings, concurrency, build hygiene
Executive Summary
The codebase is generally well-engineered with strong RAII discipline, no raw new/delete in production, and good defensive patterns (cycle detection, iterator invalidation guards). However, several significant issues were found.
Totals: 3 Critical, 10 High, 16 Medium, 20+ Low
CRITICAL (3)
- C1.
atanvariance formula is mathematically wrong (Incorrect error propagation in atan #3876) - C2.
sqrtvariance propagation divides by zero when value=0 (sqrt variance propagation divides by zero when value=0 #3878) - C3.
SubbinSizes::exclusive_scanout-of-bounds read with negative index (SubbinSizes::exclusive_scan out-of-bounds read with negative index #3879)
HIGH (10)
- H5.
ElementArrayView.__getitem__-- dangling reference risk (ElementArrayView lifetime errors in Python #3894) - H6.
ElementArrayView.__iter__-- missingkeep_alive(ElementArrayView lifetime errors in Python #3894)
H1. asin/acos variance divide-by-zero at domain boundary
lib/core/include/scipp/core/value_and_variance.h:132
At |x| == 1, arc_trig_variance divides by 1 - 1 = 0, producing inf variance. The NaN guard doesn't catch this since asin(1) = pi/2 is valid.
H2. Negative exponent integer overflow in numeric::pow
lib/common/include/scipp/common/numeric.h:106
-exponent is UB when exponent == INT64_MIN. No guard exists at the C++ level.
H3. floor_divide silently returns 0 for integer div-by-zero
lib/common/include/scipp/common/numeric.h:67
b == 0 ? 0 : ... masks a mathematical error. Python raises ZeroDivisionError for this.
H4. Rebin division by zero with zero-width bins
lib/core/include/scipp/core/element/rebin.h:49
delta / owidth where owidth=0 for duplicate bin edges.
H7. HDF5 deserialization: no size validation on shape from file
src/scipp/io/hdf5.py:314-327
shape from untrusted HDF5 files is passed directly to empty(). A crafted file could trigger OOM via extreme shapes like [2^30, 2^30].
H8. Phantom histogram in __all__, deleted reduction still listed
src/scipp/__init__.py:350,403
histogram is exported but never defined. reduction is deleted at line 177 but still in __all__. Both break from scipp import * introspection.
H9. psutil is an unused mandatory dependency
requirements/base.in:2
Listed as a core dependency, never imported anywhere in the codebase.
H10. CI supply-chain: most GitHub Actions pinned by moving tag, not SHA
.github/workflows/
Actions like actions/checkout@v4 can be force-pushed upstream. Only 2 of ~15 third-party actions are SHA-pinned. No top-level permissions: {} on most workflows.
MEDIUM (16)
- M1. Dim registry TOCTOU race (Dim registry TOCTOU race #3881)
- ~ M8.
time_pointdefault constructor leavesm_durationuninitialized (time_pointdefault constructor leavesm_durationuninitialized #3883)~ Won't do, this would breakempty.
M2. Unit aliases not thread-safe
lib/units/unit.cpp:227-231
add_unit_alias/clear_unit_aliases modify global state with no synchronization (documented but hazardous).
JLW: This is correct but difficult to fix. We don't control reading from the user-defined units directly. So to fix this, we would have to put a read lock around all unit creation. Not sure that is worth the added cost.
M3. VariableFactory global singleton has no mutation guard
lib/variable/variable_factory.cpp:89
Mutable singleton with no enforcement preventing runtime registration after init.
JLW: Technically correct, but the factory is only modified during static initialisation in C++. So there is no actual bug here.
M4. PyObject move-assignment may decrement refcount without GIL
lib/python/py_object.h:24
Defaulted move-assignment replaces an existing py::object, decrementing its refcount without acquiring the GIL.
M5. py::mod_gil_not_used() -- free-threaded Python risk
lib/python/scipp.cpp:77
Many binding functions lack explicit GIL acquisition, problematic for free-threaded Python (PEP 703).
M6. reinterpret_cast of function pointer from Python integer
lib/python/transform.cpp:17
No validation that the address points to a valid function with the expected signature.
M7. operator/ variance with zero denominator
lib/core/include/scipp/core/value_and_variance.h:276
b.value == 0 produces inf/NaN in variance with no guard.
M9. volume() and stride computation can overflow int64_t
lib/core/include/scipp/core/dimensions.h:59, lib/core/strides.cpp:20
Signed integer overflow is UB in C++. No overflow check exists for large dimension products.
M10. isclose integer subtraction overflow
lib/core/include/scipp/core/element/comparison.h:40
x - y can overflow for extreme int64_t values. abs(INT64_MIN) is also UB.
M11. XSS in HTML table rendering
src/scipp/visualization/table.py:35
Variable names and units interpolated into HTML without escaping. formatting_html.py escapes consistently; table.py does not.
M12. getattr(DType, d[key]) from untrusted dict
src/scipp/compat/dict.py:166
Should use an allowlist of valid dtype names instead of unrestricted getattr.
M13. subspan_view creates spans with no lifetime tie
lib/variable/subspan_view.cpp:18-33
Returned Variable contains std::span<T> pointing into source data. No ownership link in the C++ API.
M14. Dict::end() const passes wrong value iterator
lib/core/include/scipp/core/dict.h:370
Passes m_values.cbegin() instead of cend(). Works by accident since end iterators are never dereferenced.
M15. CMake minimum version inconsistency
CMakeLists.txt requires 3.28, CMakePresets.json claims 3.16, pyproject.toml says 3.21.
M16. Numpy constraint mismatch
pyproject.toml says >= 2 (open-ended), requirements/base.in says >=2,<2.4 (bounded).
LOW (selected highlights)
| Area | Location | Issue |
|---|---|---|
| Edge cases | lib/core/include/scipp/core/element/sort.h:33 |
std::sort with NaN violates strict weak ordering (UB per standard) |
| Edge cases | lib/core/include/scipp/core/element/math.h:74 |
reciprocal(0) produces inf with no warning |
| Memory | lib/core/include/scipp/core/memory_pool.h:59 |
MemoryPool never returns memory to OS (disabled but latent) |
| Memory | lib/core/include/scipp/core/element_array.h:154 |
Uses -1 sentinel instead of optional |
| Memory | lib/core/include/scipp/core/memory_pool.h:59-74 |
deallocate has O(n*m) linear scan per deallocation |
| Memory | lib/core/include/scipp/core/bucket_array_view.h:48-54 |
Raw pointer to buffer with no ownership |
| Memory | lib/dataset/include/scipp/dataset/data_array.h:105-107 |
Three shared_ptr members can theoretically form cycles |
| Security | src/scipp/visualization/formatting_datagroup_html.py:91 |
Broad except Exception: pass in rendering code |
| Security | src/scipp/io/hdf5.py:116 |
HDF5 read_direct potential buffer size mismatch |
| Security | src/scipp/logging.py:81 |
Log record message inserted as raw HTML |
| Concurrency | lib/python/py_object.cpp:11-42 |
PyObject GIL contention in TBB parallel loops (correct but slow) |
| Concurrency | lib/variable/include/scipp/variable/variable.h:165 |
Variable uses shared_ptr without COW; concurrent writes to shared buffers have no protection |
| Build | src/scipp/io/hdf5.py:480,532 |
Stale TODO comments for numpy<2 workarounds (numpy>=2 already required) |
| Build | across lib/ |
30+ TODO/FIXME markers including // TODO implement stubs |
| Build | .github/dependabot.yml:14 |
Dependabot only monitors plopp, ignoring all other pip deps |
| Build | .github/workflows/docs.yml:52 |
sleep 120 in docs workflow for PyPI delay |
| Build | .github/workflows/release-nightly.yml:42-53 |
Nightly wheel cleanup is commented out |
| Build | .github/workflows/coverage.yml:21-24 |
Hardcoded g++-11 may break on future Ubuntu |
| Build | .github/workflows/nightly-deps-test.yml:25 |
Uses actions/checkout@v6 while others use @v4 |
| Tests | various | Missing test coverage: logging.py, plotting.py, _binding.py, serialization.py |
| Docs | src/scipp/core/data_group.py:675 |
__invert__ docstring says "or" instead of "invert" |
| Docs | src/scipp/spatial/__init__.py:342 |
Quaternion coefficient order mismatch in docstring |
| Hygiene | across Python source | 335 # type: ignore and # noqa suppression comments across 55 files |
| Hygiene | across Python source | from __future__ import annotations used in only 24 of 91 files |
| Hygiene | lib/python/geometry.cpp:28-34 |
Deprecated rotation_matrix_from_quaternion_coeffs never removed |
| Hygiene | all source files | Copyright years frozen at 2023 |
Positive Design Patterns
Positive Design Patterns
The codebase demonstrates several strong practices worth noting:
- No raw
new/deletein production code -- all allocation goes throughunique_ptr,shared_ptr, orelement_array - Dict iterator invalidation detection -- throws if the container changes during iteration
- Deleted rvalue-qualified iterator accessors -- prevents dangling iterators from temporaries
- Cycle detection on assignment --
check_nested_in_assignrecursively preventsshared_ptrreference loops - RAII throughout -- no explicit cleanup functions or manual resource release
- Exception-safe dataset operations -- dry-run validation before modifying data
- No
eval/exec/pickle.loadson untrusted data - Data downloads use hash-verified pooch registry
- HDF5 path traversal mitigated by replacing
/in group names - GIL management is mostly correct -- release for C++ ops, re-acquire for Python callbacks
Top 10 Recommendations
Top 10 Recommendations (Priority Order)
- Fix
atanvariance formula (C1) -- Change1 - a.value * a.valueto1 + a.value * a.value - Add zero guard to
sqrtvariance (C2) -- Follow the pattern frompow(line 73-74) - Fix
SubbinSizes::exclusive_scanbounds check (C3) -- Adddelta + i < 0check - Add
keep_alivetoElementArrayView.__iter__(H6) -- Addpy::keep_alive<0, 1>() - Add HDF5 shape size validation (H7) -- Validate
product(shape)before allocating - Fix Dim registry TOCTOU (M1) -- Re-check
custom_ids().find(label)after acquiring write lock - Add HTML escaping to
table.py(M11) -- Usehtml.escape()on names/units - Remove phantom
histogramfrom__all__, fixreduction(H8) - Remove unused
psutildependency (H9) - Pin CI actions to SHA hashes (H10)
Activity
time_pointdefault constructor leavesm_durationuninitialized #3883