Skip to content

RFC: restrict table._column_mixin to CPython's limited API - #18227

Merged
pllim merged 1 commit into
astropy:mainfrom
neutrinoceros:bld/table/limited_api_compat
Jul 2, 2025
Merged

pllim merged 1 commit into
astropy:mainfrom
neutrinoceros:bld/table/limited_api_compat

Conversation

@neutrinoceros

@neutrinoceros neutrinoceros commented Jun 6, 2025 •

Copy link
Copy Markdown
Contributor

Description

Partially address #18163
I wanted to open this quickly to set an example of how many of our compile-time incompatibilities regarding the use of PyTypeObject can be resolved.

ping @astrofrog

  • By checking this box, the PR author has requested that maintainers do NOT use the "Squash and Merge" button. Maintainers should respect this when possible; however, the final decision is at the discretion of the maintainer that merges the PR.

@github-actions

github-actions Bot commented Jun 6, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

@github-actions

github-actions Bot commented Jun 6, 2025

Copy link
Copy Markdown
Contributor

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

@neutrinoceros neutrinoceros changed the title BLD: refactor table._column_mixin Cython module for compatibility with CPython's limited API BLD: refactor table._column_mixin Cython module for compatibility with CPython's limited API Jun 6, 2025
@neutrinoceros
neutrinoceros marked this pull request as ready for review June 6, 2025 16:52
@neutrinoceros
neutrinoceros requested a review from taldcroft as a code owner June 6, 2025 16:52
Comment thread astropy/table/_column_mixins.pyx Outdated
Comment thread astropy/table/_column_mixins.pyx Outdated
@neutrinoceros
neutrinoceros force-pushed the bld/table/limited_api_compat branch from a60206b to 1c693d1 Compare June 7, 2025 09:25
@neutrinoceros
neutrinoceros force-pushed the bld/table/limited_api_compat branch from 1c693d1 to 86c6a27 Compare June 9, 2025 12:31
@neutrinoceros neutrinoceros changed the title BLD: refactor table._column_mixin Cython module for compatibility with CPython's limited API RFC: restrict table._column_mixin to CPython's limited API Jun 9, 2025
@astrofrog

Copy link
Copy Markdown
Member

@neutrinoceros - now that #18236 is merged, could you rebase and remove the exceptions for astropy/table? (and similarly for the WCS PR)

@neutrinoceros
neutrinoceros force-pushed the bld/table/limited_api_compat branch 2 times, most recently from eed01a4 to 01fc1a1 Compare June 9, 2025 14:15
@neutrinoceros
neutrinoceros marked this pull request as draft June 9, 2025 14:17
@neutrinoceros
neutrinoceros force-pushed the bld/table/limited_api_compat branch from 5a6eafb to 310f16b Compare June 9, 2025 14:19
@neutrinoceros
neutrinoceros marked this pull request as ready for review June 9, 2025 14:19
@astrofrog astrofrog added the benchmark Run benchmarks for a PR label Jun 9, 2025
@astrofrog
astrofrog requested a review from mhvk June 9, 2025 16:26

@mhvk mhvk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed this one earlier - this looks like a good solution to me, and should impose little or no slowdown.

@taldcroft

Copy link
Copy Markdown
Member

@astrofrog - I have no opinion on the code itself (defer to @mhvk on that), but can you some minimal profiling to confirm that performance is not impacted?

@neutrinoceros

Copy link
Copy Markdown
Contributor Author

Benchmarks already ran successfully. Do you want something more fine grained ?

@astrofrog

Copy link
Copy Markdown
Member

For the record, here are the benchmarks we are running:

https://github.com/astropy/astropy-benchmarks/blob/main/benchmarks/table.py

which benchmark(s) is/are the most relevant to look at here? (as @neutrinoceros you can see the results in the CI job for benchmarks)

@taldcroft

Copy link
Copy Markdown
Member

I see the benchmarks ran. I downloaded the artifacts but the JSON files there are not immediately useful. Are there tools that help to analyze the data? Looking at the table tests, the closest test to getting a single item from a column is:

    def time_item_get_colfirst(self):
        self.table["b"][300]

However, this includes getting the column self.table["b"]. I guess that will be constant time, but still somewhat dilutes the impact of a change in the underlying getitem time.

In any case, I am satisified that the code does not impact item-get performance with this:

c = Column([1.5, 2.5])
%timeit c[1]

There was no appreciable difference between main and this branch, so all good!

@neutrinoceros
neutrinoceros force-pushed the bld/table/limited_api_compat branch from 310f16b to db9a0a5 Compare July 2, 2025 14:15
@neutrinoceros

Copy link
Copy Markdown
Contributor Author

rebased to refresh CI (some required jobs were not triggered).
It looks like we could merge this one as soon as it gets a formal approval ?

@taldcroft taldcroft left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@taldcroft

Copy link
Copy Markdown
Member

The Check PR change log job failed to get started. Not sure how to fix that.

@pllim
pllim force-pushed the bld/table/limited_api_compat branch from db9a0a5 to f0275be Compare July 2, 2025 16:36
@pllim

pllim commented Jul 2, 2025

Copy link
Copy Markdown
Member

There was some Ubuntu server hiccup too. I just rebased it so everything would rerun.

Is this good to merge if CI green? Thanks, all!

@pllim

pllim commented Jul 2, 2025

Copy link
Copy Markdown
Member

Benchmark failure unrelated (astropy/astropy-benchmarks#137) but also means we cannot test that in PR right now.

....time_large_gauss_combined_2d_SLSQPLSQFitter failed

@pllim
pllim merged commit ec2d974 into astropy:main Jul 2, 2025
@neutrinoceros
neutrinoceros deleted the bld/table/limited_api_compat branch July 2, 2025 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants