Skip to content

CI: Upload wheels to scipp pypi index - #3542

Merged
MridulS merged 34 commits into
mainfrom
MridulS-patch-1
Oct 4, 2024
Merged

MridulS merged 34 commits into
mainfrom
MridulS-patch-1

Conversation

@MridulS

@MridulS MridulS commented Sep 12, 2024

Copy link
Copy Markdown
Member

We can push wheels to pypi.anaconda.org/scipp and then we can use something like:

python -m pip install \
  --pre \
  --index-url https://pypi.anaconda.org/scipp/simple \
  --extra-index-url https://pypi.org/simple \
  scipp

to get nightly wheels instead of renaming built wheels and creating static urls.

We also should have something like https://github.com/scientific-python/upload-nightly-action/blob/main/.github/workflows/remove-wheels.yml to clean up old wheels, but let's test this infrastructure out first. If we are happy, we can further iterate on this :)

@jl-wynen

Copy link
Copy Markdown
Member

In your example command, why is anaconda the index-url and PyPI extra-index-url? Does the order matter?

@jl-wynen

Copy link
Copy Markdown
Member

Please explain this setup in the dev docs. E.g., https://scipp.github.io/development/releasing-scipp.html

@MridulS

MridulS commented Sep 12, 2024

Copy link
Copy Markdown
Member Author

The order does matter, with pip index-url is the main index and extra-index-url is the fallback one. But if we start using uv it actually switches this order and breaks away from pip compatibilty astral-sh/uv#2083, so with uv this should be:

python -m uv pip install \
  --pre \
  --extra-index-url https://pypi.anaconda.org/scipp/simple \
  scipp

@MridulS

MridulS commented Sep 12, 2024

Copy link
Copy Markdown
Member Author

The triggered did upload all the wheels to https://pypi.anaconda.org/scipp/simple/scipp !
But it is also uploading these random doc links !? I'm not sure where it's getting these things.

@jl-wynen

Copy link
Copy Markdown
Member

The links all point to the latest docs. I don't think they are a big issue if they simply get overwritten.

Does the action overwrite existing wheels?

@MridulS

MridulS commented Sep 12, 2024

Copy link
Copy Markdown
Member Author

Does the action overwrite existing wheels?

No, it will only overwrite if they had the same name but they won't as it uses the commit hash in the name. We would need to setup a cleaning action to make sure we don't keep the old wheels around. https://github.com/scientific-python/upload-nightly-action/blob/main/.github/workflows/remove-wheels.yml as an example.

@jl-wynen

Copy link
Copy Markdown
Member

This looks pretty complicated. Do we really need all of this? Or is there a simple way to wipe the index every night before upload?

Comment thread .github/workflows/release-nightly.yml Outdated
Comment thread .github/workflows/release-nightly.yml Outdated
Comment thread .github/workflows/release-nightly.yml Outdated
Comment thread .github/workflows/release-nightly.yml Outdated
@MridulS

MridulS commented Sep 27, 2024

Copy link
Copy Markdown
Member Author

I'm waiting for a new release upstream https://github.com/scientific-python/upload-nightly-action first before merging this.

Comment thread .github/workflows/release-nightly.yml Outdated
Comment thread .github/workflows/release-nightly.yml
@MridulS
MridulS enabled auto-merge (squash) October 1, 2024 09:50
Comment thread .github/workflows/release-nightly.yml Outdated
Comment thread .github/workflows/release-nightly.yml Outdated
Comment thread .github/workflows/release-nightly.yml Outdated
Comment thread .github/workflows/release-nightly.yml Outdated
@MridulS
MridulS merged commit 54df1b8 into main Oct 4, 2024
@MridulS
MridulS deleted the MridulS-patch-1 branch October 4, 2024 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants