Use huge pages for large buffers on Linux - #3937

It was always been possible to use Scipp with hugepages (see https://scipp.github.io/user-guide/tips-tricks-and-anti-patterns.html#Using-HugePages), but it required user intervention and adequate system settings, which required root permissions.

Scipp allocates its buffers with plain new T[], which never calls madvise(MADV_HUGEPAGE). On systems with transparent_hugepage=madvise — the default on many distributions, and the case users hit on HPC clusters where they cannot change the setting — Scipp therefore runs on 4 kByte pages, while NumPy in the same process uses huge pages, since it madvises every array of 4 MByte and more. Setting transparent_hugepage=always was the only documented workaround, and it needs root.

Neither libtbbmalloc_proxy.so nor TBB_MALLOC_USE_HUGE_PAGES help: tbbmalloc has no madvise call at all and treats huge pages as unavailable unless they are reserved explicitly or the kernel setting is always. The claim in our docs that the madvise setting "is not sufficient for Scipp" was accurate, but the reason was on our side, and it costs up to 1.7x on memory-bound operations.

Buffers of 4 MByte and more now request huge pages, matching NumPy's threshold. Below that the kernel would round up to the 2 MByte huge page size and waste memory. SCIPP_MADVISE_HUGEPAGE=0 disables it, which may be needed on systems suffering from memory fragmentation: with the common defrag=madvise setting the kernel compacts memory synchronously when a page of an advised range is first touched, which can stall the allocating thread.

Reported in https://github.com/orgs/scipp/discussions/3936.

Benchmarks

Operations on 1 GByte buffers, best of 3, on a system with transparent_hugepage=madvise, defrag=madvise, glibc 2.35, kernel 6.8:

Both columns are the same build, the first one run with SCIPP_MADVISE_HUGEPAGE=0, i.e., with the behavior before this change.

The script used for this, and the raw output including a comparison with libtbbmalloc_proxy.so and GLIBC_TUNABLES=glibc.malloc.hugetlb=1, are in https://gist.github.com/SimonHeybrock/132292e373059e68f6ac970524caeb23.

Test plan

The added unit test allocates a large buffer and checks VmFlags in /proc/self/smaps for the hg flag, i.e., that the kernel recorded the advice. This works independently of whether the system can actually provide huge pages, so it is meaningful in containers, and it skips on kernels built without huge page support.

🤖 Generated with Claude Code