Skip to content

Claude code review #3875

Open
6 of 6 issues completed
Open

Description

@jl-wynen

@MridulS asked Claude for a code review and here is the result.
We should check each point to see if it is valid and if so, fix it.

Scipp Codebase Deep Audit Report

Date: 2026-04-09
Scope: ~411 C++ files, ~91 Python source files, ~93 test files
Areas: Memory management, edge cases/UB, Python security, pybind11 bindings, concurrency, build hygiene

Executive Summary

The codebase is generally well-engineered with strong RAII discipline, no raw new/delete in production, and good defensive patterns (cycle detection, iterator invalidation guards). However, several significant issues were found.

Totals: 3 Critical, 10 High, 16 Medium, 20+ Low


CRITICAL (3)


HIGH (10)

H1. asin/acos variance divide-by-zero at domain boundary

lib/core/include/scipp/core/value_and_variance.h:132

At |x| == 1, arc_trig_variance divides by 1 - 1 = 0, producing inf variance. The NaN guard doesn't catch this since asin(1) = pi/2 is valid.

H2. Negative exponent integer overflow in numeric::pow

lib/common/include/scipp/common/numeric.h:106

-exponent is UB when exponent == INT64_MIN. No guard exists at the C++ level.

H3. floor_divide silently returns 0 for integer div-by-zero

lib/common/include/scipp/common/numeric.h:67

b == 0 ? 0 : ... masks a mathematical error. Python raises ZeroDivisionError for this.

H4. Rebin division by zero with zero-width bins

lib/core/include/scipp/core/element/rebin.h:49

delta / owidth where owidth=0 for duplicate bin edges.

H7. HDF5 deserialization: no size validation on shape from file

src/scipp/io/hdf5.py:314-327

shape from untrusted HDF5 files is passed directly to empty(). A crafted file could trigger OOM via extreme shapes like [2^30, 2^30].

H8. Phantom histogram in __all__, deleted reduction still listed

src/scipp/__init__.py:350,403

histogram is exported but never defined. reduction is deleted at line 177 but still in __all__. Both break from scipp import * introspection.

H9. psutil is an unused mandatory dependency

requirements/base.in:2

Listed as a core dependency, never imported anywhere in the codebase.

H10. CI supply-chain: most GitHub Actions pinned by moving tag, not SHA

.github/workflows/

Actions like actions/checkout@v4 can be force-pushed upstream. Only 2 of ~15 third-party actions are SHA-pinned. No top-level permissions: {} on most workflows.


MEDIUM (16)

M2. Unit aliases not thread-safe

lib/units/unit.cpp:227-231

add_unit_alias/clear_unit_aliases modify global state with no synchronization (documented but hazardous).

JLW: This is correct but difficult to fix. We don't control reading from the user-defined units directly. So to fix this, we would have to put a read lock around all unit creation. Not sure that is worth the added cost.

M3. VariableFactory global singleton has no mutation guard

lib/variable/variable_factory.cpp:89

Mutable singleton with no enforcement preventing runtime registration after init.

JLW: Technically correct, but the factory is only modified during static initialisation in C++. So there is no actual bug here.

M4. PyObject move-assignment may decrement refcount without GIL

lib/python/py_object.h:24

Defaulted move-assignment replaces an existing py::object, decrementing its refcount without acquiring the GIL.

M5. py::mod_gil_not_used() -- free-threaded Python risk

lib/python/scipp.cpp:77

Many binding functions lack explicit GIL acquisition, problematic for free-threaded Python (PEP 703).

M6. reinterpret_cast of function pointer from Python integer

lib/python/transform.cpp:17

No validation that the address points to a valid function with the expected signature.

M7. operator/ variance with zero denominator

lib/core/include/scipp/core/value_and_variance.h:276

b.value == 0 produces inf/NaN in variance with no guard.

M9. volume() and stride computation can overflow int64_t

lib/core/include/scipp/core/dimensions.h:59, lib/core/strides.cpp:20

Signed integer overflow is UB in C++. No overflow check exists for large dimension products.

M10. isclose integer subtraction overflow

lib/core/include/scipp/core/element/comparison.h:40

x - y can overflow for extreme int64_t values. abs(INT64_MIN) is also UB.

M11. XSS in HTML table rendering

src/scipp/visualization/table.py:35

Variable names and units interpolated into HTML without escaping. formatting_html.py escapes consistently; table.py does not.

M12. getattr(DType, d[key]) from untrusted dict

src/scipp/compat/dict.py:166

Should use an allowlist of valid dtype names instead of unrestricted getattr.

M13. subspan_view creates spans with no lifetime tie

lib/variable/subspan_view.cpp:18-33

Returned Variable contains std::span<T> pointing into source data. No ownership link in the C++ API.

M14. Dict::end() const passes wrong value iterator

lib/core/include/scipp/core/dict.h:370

Passes m_values.cbegin() instead of cend(). Works by accident since end iterators are never dereferenced.

M15. CMake minimum version inconsistency

CMakeLists.txt requires 3.28, CMakePresets.json claims 3.16, pyproject.toml says 3.21.

M16. Numpy constraint mismatch

pyproject.toml says >= 2 (open-ended), requirements/base.in says >=2,<2.4 (bounded).


LOW (selected highlights)

Area Location Issue
Edge cases lib/core/include/scipp/core/element/sort.h:33 std::sort with NaN violates strict weak ordering (UB per standard)
Edge cases lib/core/include/scipp/core/element/math.h:74 reciprocal(0) produces inf with no warning
Memory lib/core/include/scipp/core/memory_pool.h:59 MemoryPool never returns memory to OS (disabled but latent)
Memory lib/core/include/scipp/core/element_array.h:154 Uses -1 sentinel instead of optional
Memory lib/core/include/scipp/core/memory_pool.h:59-74 deallocate has O(n*m) linear scan per deallocation
Memory lib/core/include/scipp/core/bucket_array_view.h:48-54 Raw pointer to buffer with no ownership
Memory lib/dataset/include/scipp/dataset/data_array.h:105-107 Three shared_ptr members can theoretically form cycles
Security src/scipp/visualization/formatting_datagroup_html.py:91 Broad except Exception: pass in rendering code
Security src/scipp/io/hdf5.py:116 HDF5 read_direct potential buffer size mismatch
Security src/scipp/logging.py:81 Log record message inserted as raw HTML
Concurrency lib/python/py_object.cpp:11-42 PyObject GIL contention in TBB parallel loops (correct but slow)
Concurrency lib/variable/include/scipp/variable/variable.h:165 Variable uses shared_ptr without COW; concurrent writes to shared buffers have no protection
Build src/scipp/io/hdf5.py:480,532 Stale TODO comments for numpy<2 workarounds (numpy>=2 already required)
Build across lib/ 30+ TODO/FIXME markers including // TODO implement stubs
Build .github/dependabot.yml:14 Dependabot only monitors plopp, ignoring all other pip deps
Build .github/workflows/docs.yml:52 sleep 120 in docs workflow for PyPI delay
Build .github/workflows/release-nightly.yml:42-53 Nightly wheel cleanup is commented out
Build .github/workflows/coverage.yml:21-24 Hardcoded g++-11 may break on future Ubuntu
Build .github/workflows/nightly-deps-test.yml:25 Uses actions/checkout@v6 while others use @v4
Tests various Missing test coverage: logging.py, plotting.py, _binding.py, serialization.py
Docs src/scipp/core/data_group.py:675 __invert__ docstring says "or" instead of "invert"
Docs src/scipp/spatial/__init__.py:342 Quaternion coefficient order mismatch in docstring
Hygiene across Python source 335 # type: ignore and # noqa suppression comments across 55 files
Hygiene across Python source from __future__ import annotations used in only 24 of 91 files
Hygiene lib/python/geometry.cpp:28-34 Deprecated rotation_matrix_from_quaternion_coeffs never removed
Hygiene all source files Copyright years frozen at 2023

Positive Design Patterns

Positive Design Patterns

The codebase demonstrates several strong practices worth noting:

  • No raw new/delete in production code -- all allocation goes through unique_ptr, shared_ptr, or element_array
  • Dict iterator invalidation detection -- throws if the container changes during iteration
  • Deleted rvalue-qualified iterator accessors -- prevents dangling iterators from temporaries
  • Cycle detection on assignment -- check_nested_in_assign recursively prevents shared_ptr reference loops
  • RAII throughout -- no explicit cleanup functions or manual resource release
  • Exception-safe dataset operations -- dry-run validation before modifying data
  • No eval/exec/pickle.loads on untrusted data
  • Data downloads use hash-verified pooch registry
  • HDF5 path traversal mitigated by replacing / in group names
  • GIL management is mostly correct -- release for C++ ops, re-acquire for Python callbacks

Top 10 Recommendations

Top 10 Recommendations (Priority Order)

  1. Fix atan variance formula (C1) -- Change 1 - a.value * a.value to 1 + a.value * a.value
  2. Add zero guard to sqrt variance (C2) -- Follow the pattern from pow (line 73-74)
  3. Fix SubbinSizes::exclusive_scan bounds check (C3) -- Add delta + i < 0 check
  4. Add keep_alive to ElementArrayView.__iter__ (H6) -- Add py::keep_alive<0, 1>()
  5. Add HDF5 shape size validation (H7) -- Validate product(shape) before allocating
  6. Fix Dim registry TOCTOU (M1) -- Re-check custom_ids().find(label) after acquiring write lock
  7. Add HTML escaping to table.py (M11) -- Use html.escape() on names/units
  8. Remove phantom histogram from __all__, fix reduction (H8)
  9. Remove unused psutil dependency (H9)
  10. Pin CI actions to SHA hashes (H10)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions