Repository navigation
Delocate error when building Pillow on macOS #8671
Description
Activity
DYLD_LIBRARY_PATHwas indeed added to pyproject.toml in #8500, and if I removeDYLD_LIBRARY_PATHfrom pyproject.toml, I see the error that you're reporting - https://github.com/radarhere/Pillow/actions/runs/12664298313/job/35292210395#step:5:8924I wonder if you're hitting https://cibuildwheel.pypa.io/en/stable/faq/#macos-passing-dyld_library_path-to-delocate, and the solution is radarhere@27dd70a
diff --git a/pyproject.toml b/pyproject.toml index 2c6c7bcd0..4b9bdd5d6 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -96,15 +96,17 @@ before-all = ".github/workflows/wheels-dependencies.sh" build-verbosity = 1 config-settings = "raqm=enable raqm=vendor fribidi=vendor imagequant=disable" -# Disable platform guessing on macOS -macos.config-settings = "raqm=enable raqm=vendor fribidi=vendor imagequant=disable platform-guessing=disable" test-command = "cd {project} && .github/workflows/wheels-test.sh" test-extras = "tests" +[tool.cibuildwheel.macos] +repair-wheel-command = "DYLD_LIBRARY_PATH=$(pwd)/build/deps/darwin/lib delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}" +# Disable platform guessing on macOS +config-settings = "raqm=enable raqm=vendor fribidi=vendor imagequant=disable platform-guessing=disable" + [tool.cibuildwheel.macos.environment] PATH = "$(pwd)/build/deps/darwin/bin:$(dirname $(which python3)):/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" -DYLD_LIBRARY_PATH = "$(pwd)/build/deps/darwin/lib" [tool.black] exclude = "wheels/multibuild"
@radarhere That does appear to the cluster of issues that I'm hitting.
I didn't know that GitHub Actions had disabled SIP, but there's some suggestion that it has been disabled on the GitHub Actions images as of macOS-13. That would explain why this problem isn't occurring in CI.
It surprises me a little that I'm the first person to hit this - but given the zlib_ng patch doesn't have any macOS-specific pieces, and the CI config works (producing viable wheels), I might just be very lucky in being the first person to try rebuilding the whole of Pillow on macOS since the zlib-ng PR landed.
Regardless - As for a fix - the approach you've described would work - this is the other possible approach:
diff --git a/.github/workflows/wheels-dependencies.sh b/.github/workflows/wheels-dependencies.sh index d721c750c..3c9cc20f9 100755 --- a/.github/workflows/wheels-dependencies.sh +++ b/.github/workflows/wheels-dependencies.sh @@ -139,6 +139,14 @@ function build_zlib_ng { && ./configure --prefix=$BUILD_PREFIX --zlib-compat \ && make -j4 \ && make install) + + if [ -n "$IS_MACOS" ]; then + # Ensure that on macOS, the library name is an absolute path, not an + # @rpath, so that delocate picks up the right library (and doesn't need + # DYLD_LIBRARY_PATH to be set). The default Makefile doesn't have an + # option to control the install_name. + install_name_tool -id $BUILD_PREFIX/lib/libz.1.dylib $BUILD_PREFIX/lib/libz.1.dylib + fi touch zlib-stamp }
The argument I'd make for this approach is that it doesn't require redefining "default" behavior of cibuildwheel. However, I could go either way. If you'd like me to turn this diff into a PR, let me know.
@radarhere I couldn't help myself - see #8673 - but it looks like you've gone the explicit route with #8672.
What did you do?
When trying to build macOS wheels with cibuildwheel on a personal machine, an error is raised during the delocate step.
What did you expect to happen?
A cibuildwheel build should successfully complete.
What actually happened?
What are your OS, Python and Pillow versions?
Additional details
The problem appears to have been caused by the introduction of zlib-ng. On my 2 test machines, the compiled version of this library has an
install_nameof@rpath/libz.1.dylib. This is fine, but it requires thatDYLD_LIBRARY_PATHincludes the build dependencies path.When cibuildwheel invokes
delocate-wheel,DYLD_LIBRARY_PATHis set to include the build dependencies lib directory. However,delocate-wheelis invoked withshell(), and macOS does not passDYLD_LIBRARY_PATHto child subprocesses.All the other binary dependencies appear to be compiled with absolute paths in the
install_name.This clearly isn't an issue with CI, as wheel builds are succeeding there. However, I can't work out why they are succeeding. I can only assume that there is something on the CI machine that puts a
libz.1.dylibcompatible library on the linking path which is able to satisfy delocate.I've been able to work around the issue locally by manually rewriting the install name of libz.dylib after the build-zlib-ng completes (there doesn't appear to be a compilation option to explicitly guarantee an absolute install_name). However, it's not clear to me if I've missed something obvious in my setup, or if this is an issue that others are likely to see.