Repository navigation
Create py_zipapp_binary to for building zip from py_binary #2586
Description
Right now, py_binary has an implicit output ({name}.zip}) and output group (python_zip_file) that builds a zip file of the py_binary. This zip file is a "zipapp", basically a zip file with a top-level __main__.py file. See https://peps.python.org/pep-0441/ and https://docs.python.org/3/library/zipapp.html for details.
If the Bazel builtin flag --build_python_zip is set, then the zip output changes slightly: it is turned into a self-executable zip file by prepending some shell code to the zip file.
Having this part of py_binary itself is problematic for four main reasons:
- It makes maintaining py_binary harder. The logic for creating the zip is subtle and introduces several branches on code that is otherwise straight forward.
- A zipapp is just one of many ways to create a "distributable" py_binary. It doesn't make sense to bake one directly into py_binary.
- A zipapp only works if it's pure-python code. If there are C library dependencies, then the zip must be extracted, which then breaks the logic for handling zipapp-based startup.
- A zip file doesn't have native support for symlinks. This makes it incompatible with bootstrap=script and how it creates a venv with symlinks to the desired python interpreter. Working around this requires hacks being applied at runtime with their own problems.
What to do:
- Create a
py_zipapp_binaryandpy_zipapp_testrule. As input, they take a py_binary As output, they produce a zipapp. An attribute decides if its executable or not. - As part of the above: create a tool to generate zip files that contain symlinks. Use
File.is_symlinkto detect if inputs should be symlinks or not. For a python-based implementation, how to create a symlink: https://stackoverflow.com/questions/35782941/archiving-symlinks-with-python-zipfile . It doesn't necessairily have to be implemented in Python. Another language is fine, however, if it's a compiled language, then we'll need to update rules_python handle building and consuming prebuilt artifacts. - Add windows support to py_zipapp_binary et al
- Add python_zip_file output group to py_zipapp for compatibility
- Add deprecation warning for py_binary zipapp usage
- Add entries to features.bzl
- Default windows to non-zip builds
- Disable and remove the
--build_python_zipflag andpython_zip_fileoutput group.
I'll note that there are rule sets, e.g. rules_pkg that support somewhat arbitrary creation of zip files. Using one of those would also be fine.
Activity
arrdem commentedon Feb 17, 2025
This is almost exactly the strategy that my rules_zapp implementation used, for all it was the first ruleset I wrote. I'd be open to contributing something along those lines if that's of interest.
rickeylev commentedon Feb 17, 2025
Yes, that sounds pretty promising!
feat: add py_zipapp_binary and test rules for zipapp support (#3539)
feat(zipapp): add windows support (#3561)
rickeylev commentedon Feb 6, 2026
Hrm. Something tricky: changing the default of build_python_zip to False for windows isn't directly possible because the flag definition is built into Bazel pre-bazel 9.
Maybe: for windows, stop using the builtin build_python_zip flag, and instead require setting the starlark flag? And then default it to False for windows, too.
feat(zipapp): Add python_zip_file output group for better compatibili…
feat(zipapp): Add python_zip_file output group for better compatibili…
chore(py_executable): print warning if build zip is enabled (#3568)
rickeylev commentedon Feb 9, 2026
Another complication for windows: I think one of the reasons a zip file was used for windows is because runfiles trees aren't created by default for it. It uses a "manifest only" mode, where a file has a mapping of the runfiles.
I think what this means is
startup --windows_enable_symlinksbecomes a required setting. Or force enabling the--build_runfile_links=true.I'm going to try having the transition force-enable build_runfile_links and see if that is a viable work around.
feat(zipapp): add windows support (bazel-contrib#3561)
feat(zipapp): Add python_zip_file output group for better compatibili…
chore(py_executable): print warning if build zip is enabled (bazel-co…
BarrettStephen commentedon May 13, 2026
@rickeylev, we tried to upgrade to rules_python 2.0.1 and ran into this issue breaking our windows builds in CI and thus all of our windows users. Getting anyone and everyone on windows to have symlink privledge is quite burdensome. This is why
build_runfile_linksis false on windows in the first place. Have you given this more thought?rickeylev commentedon May 13, 2026
I gave it thought, but really, it comes down to available time/effort. A non-symlink based venv is a substantially different implementation, so the net effect is supporting Windows requires a lot more work. We'd gladly accept a maintainer who was interested in Windows support. Neither I or Ignas deal with Windows on a regular basis, so the startup and context switch cost is pretty high.
If someone is running Bazel, I don't see why it would be a burden? As back as far as Windows 10 allows enabling symlinks via developer mode (which doesn't require admin, iiuc).
Are you talking more about deployment of a zipapp? I could see that being problematic. Having the zipapp rules support a non-symlink-based output would be much more feasible.
BarrettStephen commentedon May 13, 2026
I work at a large company that supports several operating systems due to a diverse set of users. It is unfortunate that taking a new version of the ubiquitous rules_python breaks windows by default and that maintaining windows support is not something the maintainers are interested in.
I'll explore my options with a companywide IT change, but I'm concerned about the long term here if Windows functionality is not verified as rules_python evolves.
It looks like https://github.com/periareon/rules_venv doesn't have this issue, but I'm likely to need a rules_python solution anyway due to other deps.
sputt commentedon May 13, 2026
I am hitting this too.
To be clear, this is not just
py_zipapp_binaryimpacted, this ispy_binaryin general. The latestrules_python2.0.1 shows:Unless I am mistaken, there doesn't seem to be a way to opt out of this anymore.