Repository navigation
Release Windows Store app containing Python #79158
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/zooba'
closed_at = <Date 2018-12-12.23:45:02.538>
created_at = <Date 2018-10-13.22:43:47.118>
labels = ['3.8', 'type-feature', '3.7', 'OS-windows']
title = 'Release Windows Store app containing Python'
updated_at = <Date 2019-10-18.00:21:36.712>
user = 'https://github.com/zooba'bugs.python.org fields:
activity = <Date 2019-10-18.00:21:36.712>
actor = 'izbyshev'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2018-12-12.23:45:02.538>
closer = 'steve.dower'
components = ['Windows']
creation = <Date 2018-10-13.22:43:47.118>
creator = 'steve.dower'
dependencies = []
files = []
hgrepos = []
issue_num = 34977
keywords = ['patch']
message_count = 42.0
messages = ['327679', '327691', '328146', '328147', '330166', '330359', '330452', '330643', '331265', '331266', '331267', '331300', '331301', '331304', '331310', '331311', '331316', '331321', '331322', '331328', '331350', '331352', '331356', '331357', '331358', '331363', '331368', '331377', '331387', '331517', '331518', '331519', '331520', '331521', '331530', '331534', '331553', '331560', '331566', '331571', '331572', '331573']
nosy_count = 11.0
nosy_names = ['paul.moore', 'vinay.sajip', 'vstinner', 'tim.golden', 'jkloth', 'ned.deily', 'zach.ware', 'steve.dower', 'ethan smith', 'miss-islington', 'xtreak']
pr_nums = ['10245', '10677', '11019', '11021', '11027', '11028', '11029', '11030', '11092', '11093', '5812']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue34977'
versions = ['Python 3.7', 'Python 3.8']
Activity
zooba commentedon Oct 13, 2018
The Windows Store can now install unrestricted apps, which means we can provide the Python interpreter in there.
Advantages:
Disadvantages:
I think for a (very significant) subset of our users, this will be a much better experience than downloading our current installer. It also has the advantage of making "Python" appear in searches in the start menu, which will link directly to a one-click "Install" button.
The biggest issue is likely to be pip not installing with --user by default, as well as subsequent issues with apps that require users to navigate to their %AppData% directories manually (since these will be redirected to different locations).
But until we get a package available for testing, it will be hard to figure out what needs fixing. Given this is just distribution and not a new platform, I plan to enable it for Python 3.7 (probably 3.7.2, but it may be possible to bundle 3.7.1 as-is for testing).
tirkarthi commentedon Oct 14, 2018
+1 for reliable installation. There were some cases (bpo-33220, bpo-30944) where the installer was detected as a Virus by some vendors which would be reduced since this is installed from the App store as a verified app. I hope this will also reduce the scenarios where users need to resort to bug tracker with log files when there are installation failures [1] providing a more smooth one click install.
There is a long standing open issue [2] to make
pip install --useras a default which I hope gets resolved and there is an issue with usingpip install --userinside a virtualenv which seems to block the earlier issue [3]Thanks a lot for this!
[1] https://bugs.python.org/issue?%40columns=id%2Cactivity%2Ctitle%2Ccreator%2Cassignee%2Cstatus%2Ctype&%40sort=-activity&%40filter=status&%40action=searchid&ignore=file%3Acontent&%40search_text=windows+msi+install&submit=search&status=-1%2C1%2C2%2C3
[2] pypa/pip#1668
[3] pypa/pip#5702
zooba commentedon Oct 20, 2018
I'm 99% sure that with bpo-34725 I'll be able to make "pip --user" the default when using this install, as well as making the commands globally available (i.e. "python[3[.8]]", "pip[3[.8]]", "idle[3[.8]]" will work everywhere).
zooba commentedon Oct 20, 2018
(And I'll deal with that 1% at the PyPA sprints next weekend. We are due for a new pip insertion anyway, so I guess that'll come for 3.7.2.)
zooba commentedon Nov 21, 2018
For people watching the bug, this is ready for review.
Most of the change is adding a "layout" script (roughly the equivalent of "make install" for Windows, but with more flexibility to generate different structures, precompile stuff, etc.)
About the only CPython change is to venv. I added a new launcher variation that looks for pyvenv.cfg and runs the python.exe listed in that, so now we don't have to copy any DLLs into the venv at all (it's needed because the old one would try to LoadLibrary DLLs inside the store package, which is not allowed, but it also fixes my #1 complaint about venv which is that upgrading the base Python breaks all venvs).
There are also a few tweaks to the startup process, including extending the existing __PYVENV_LAUNCHER__ env variable from macOS to Windows as well, though it's handled in a different location. And a couple of minor tweaks to tests - issues discovered now that it's pretty easy to generate a "real" layout to run tests in.
I can break this up into a couple of PRs if that's preferred, though it doesn't really gain anything. If you're not interested in the new layout script, stop reading the PR after you finish launcher.c :)
zooba commentedon Nov 23, 2018
If anyone would like to try this out early, here's some instructions for a build I just made.
Enable sideloading apps on your machine (see https://docs.microsoft.com/en-us/windows/uwp/get-started/enable-your-device-for-development - may already be done, sideload or developer mode should do)
In a regular (non-admin) PowerShell prompt in a temporary directory, run these commands:
iwr "https://dev.azure.com/Python/8e426817-76c0-4b99-ba9e-a48a1e4bd5db/_apis/build/builds/34540/artifacts?artifactName=appx&api-version=5.0-preview.5&%24format=zip" -outfile python37.zip
Expand-Archive .\python37.zip .
Add-AppxPackage appx\python.msix
(The first may take a while as it downloads approx. 80MB. I'm intending to leave the test suite out of the final package, which should bring it down closer to 40MB.)
Now play with it:
$> python.exe
$> python3.exe
$> python3.7.exe
$> pip.exe
$> pip3.exe
$> pip3.7.exe
$> idle.exe
$> idle3.exe
$> idle3.7.exe
(Or press Win+S and search for new entries)
To remove it:
Remove-AppxPackage (Get-AppxPackage *Python.3.7)
(delete the downloaded files)
Right now, the py.exe launcher can't find it, nor can any tools that rely on enumerating the registry - directly requesting the reg keys will return the right values though. VS Code seems to pick it up just fine though, I guess because it's on PATH.
zooba commentedon Nov 26, 2018
For my own reference, some issues I found installing this package on a clean machine:
zooba commentedon Nov 29, 2018
Updated testing instructions - this one should work with just sideloading permissions I think.
iwr "https://dev.azure.com/Python/8e426817-76c0-4b99-ba9e-a48a1e4bd5db/_apis/build/builds/34760/artifacts?artifactName=appx&api-version=5.0-preview.5&%24format=zip" -outfile python37.zip
Expand-Archive .\python37.zip .
Add-AppxPackage appx\python.msix
To remove:
Remove-AppxPackage (Get-AppxPackage *Python.3.7)
(delete the downloaded files)
zooba commentedon Dec 7, 2018
New changeset 468a15a by Steve Dower in branch 'master':
bpo-34977: Add Windows App Store package (GH-10245)
468a15a
zooba commentedon Dec 7, 2018
New changeset 2532091 by Steve Dower in branch '3.7':
[3.7] bpo-34977: Add Windows App Store package (GH-10245)
2532091
21 remaining items
zooba commentedon Dec 10, 2018
I'm just waiting on Victor to sign off on PR 11029 (or someone else to do it) so I can merge it and rebase PR 11027.
Hopefully then someone will review it today so that we don't have to hold up 3.7.2rc1 any longer.
vstinner commentedon Dec 10, 2018
Sorry, I don't have the bandwidth to provide a proper review.
zooba commentedon Dec 10, 2018
Okay, I'll merge the first part now so I can rebase the main one and *hopefully* someone will be willing to review it now it's smaller. Otherwise, I think we're best to go with buildbots and all the testing I've already done.
This is my sole job today, so I'll keep an eye on the buildbots. I'll also get the docs updated as Paul suggested for the main PRs.
zooba commentedon Dec 10, 2018
New changeset 1c3de54 by Steve Dower in branch 'master':
bpo-34977: Use venv redirector instead of original python.exe on Windows (GH-11029)
1c3de54
zooba commentedon Dec 10, 2018
New changeset b264c60 by Steve Dower in branch '3.7':
[3.7] bpo-34977: Use venv redirector instead of original python.exe on Windows (GH-11029)
b264c60
jkloth commentedon Dec 10, 2018
See also bpo-35450: venv module doesn't create a copy of python binary by default
zooba commentedon Dec 10, 2018
Paul (and anyone else) - the below link should go directly to just the commit with the docs update. I did a slight rearrangement of the install docs to make the options clearer, and wrote up *just* enough info on nuget to help people use it right (I hope). Similarly for the Store package.
Any comments appreciated. Still aiming to get this in for 3.7.2rc1 today.
7d7ddbc
I've also kicked off a custom buildbot run against the 3.7 version of the change. The version in master ran earlier and was clean. Plus I'm running test builds on the release machine, so it should be ready to go.
zooba commentedon Dec 11, 2018
Thanks, Paul. Appreciate it :)
Doing a final call before I hit merge in an hour or so. Right now:
Any last pushback?
zooba commentedon Dec 11, 2018
New changeset 0cd6391 by Steve Dower in branch 'master':
bpo-34977: Add Windows App Store package (GH-11027)
0cd6391
zooba commentedon Dec 11, 2018
New changeset 0e4ad88 by Steve Dower in branch '3.7':
bpo-34977: Add Windows App Store package (GH-11027)
0e4ad88
zooba commentedon Dec 11, 2018
Luckily, just noticed that I introduced a bug as part of splitting up the original PR into two.
It wasn't going to have an impact until the package was properly installed from the store, but without the proper directory resolution in there (since the variable I removed is never defined now), in some scenarios it wouldn't be possible to launch Python from a process outside of the app package, such as a virtual environment.
Testing in a proper app container is difficult, but I'll see if I can come up with a way to do it. It might require a custom buildbot, so I'll see whether there's a way I can do that (historically we've not been able to wire internal work machines - including Azure hosted ones - into other CI systems, but I'll try again).
zooba commentedon Dec 11, 2018
New changeset d5a6a38 by Steve Dower in branch 'master':
bpo-34977: Remove unused preprocessor definition (GH-11092)
d5a6a38
miss-islington commentedon Dec 11, 2018
New changeset 9bb306d by Miss Islington (bot) in branch '3.7':
bpo-34977: Remove unused preprocessor definition (GH-11092)
9bb306d