DOC: Bump Python version on RTD build and fix failure by pllim · Pull Request #10524 · astropy/astropy

pllim · 2020-07-02T04:03:56Z

Description

RTD complained about astropy/table/table.py:docstring of astropy.table.TableColumns.keys::py:class reference target not found: a set-like object providing a view on D's keys on latest build even though the PR merged reported successful RTD build. Hoping that bumping the Python version will fix the problem if it is due to some incompatibilities with older dependencies that cannot be upgraded because Python being used is too old.

UPDATE: I think the failure is real. I think we inherited something from Python that doesn't play well with Sphinx.

Looks like there is also astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.clear::py:class reference target not found: None. Remove all items from D..

UPDATE: Comparing the logs from last successful build and the failed build on master branch, I noticed the following:

numpydoc 1.0.0 -> 1.1.0 (biggest suspect!) -- Let's pin until we get an answer from Sphinx emits "WARNING: py:class reference target not found" with numpydoc 1.1.0 numpy/numpydoc#275
pillow 7.1.2 -> 7.2.0
sphinx-build removed -E option -- Don’t use a saved environment (the structure caching all cross-references), but rebuild it completely. The default is to only read and parse source files that are new or have changed since the last run.

UPDATE: Tried to pin using docs/requirements.txt and have it installed via .readthedocs.yml before editable install part but didn't work. A later download somehow upgraded it back to 1.1.0.

TODO:

After this is merged, open a follow-up issue to unpin when Sphinx emits "WARNING: py:class reference target not found" with numpydoc 1.1.0 numpy/numpydoc#275 is resolved.

pllim · 2020-07-02T14:48:00Z

Now it is astropy/wcs/__init__.py:docstring of astropy.wcs.Wcsprm.get_cdelt::py:class reference target not found: double array[naxis]. Oh gosh, I hope I don't have to go patch CPython.

pllim · 2020-07-03T00:28:08Z

Will switching from numpydoc to napoleon fix this? Has been on our wish list over at astropy/sphinx-astropy#5 for a while now.

UPDATE: Wow, that opened a different can of worms. Maybe similar to problem at scipy/scipy#3915 .
astropy/coordinates/baseframe.py:docstring of astropy.coordinates.BaseCoordinateFrame:42:Inline strong start-string without end-string

pllim · 2020-07-03T01:22:35Z

Found some interesting tidbits at https://stackoverflow.com/questions/11417221/sphinx-autodoc-gives-warning-pyclass-reference-target-not-found-type-warning but doesn't seem to help much in this case.

pllim · 2020-07-03T01:36:29Z

Well, the switch to napoleon didn't work out, but after getting through some hurdle to build doc locally, here is a full list of warnings that RTD will encounter with the current setup on master:

astropy/table/table.py:docstring of astropy.table.TableColumns.keys:: WARNING: py:class reference target not found: a set-like object providing a view on D's keys
astropy/table/table.py:docstring of astropy.table.TableColumns.values:: WARNING: py:class reference target not found: an object providing a view on D's values
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.clear:: WARNING: py:class reference target not found: None.  Remove all items from D.
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.copy:: WARNING: py:class reference target not found: a shallow copy of D
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.items:: WARNING: py:class reference target not found: a set-like object providing a view on D's items
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.keys:: WARNING: py:class reference target not found: a set-like object providing a view on D's keys
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.pop:: WARNING: py:class reference target not found: v, remove specified key and return the corresponding value.
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.update:: WARNING: py:class reference target not found: None.  Update D from dict/iterable E and F.
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.values:: WARNING: py:class reference target not found: an object providing a view on D's values
astropy/wcs/__init__.py:docstring of astropy.wcs.Wcsprm.get_cdelt:: WARNING: py:class reference target not found: double array[naxis]
astropy/wcs/__init__.py:docstring of astropy.wcs.Wcsprm.get_pc:: WARNING: py:class reference target not found: double array[naxis][naxis]
astropy/wcs/__init__.py:docstring of astropy.wcs.Wcsprm.get_ps:: WARNING: py:class reference target not found: list of tuples
astropy/wcs/__init__.py:docstring of astropy.wcs.Wcsprm.get_pv:: WARNING: py:class reference target not found: list of tuples

Aha! numpydoc 1.1.0 broke our RTD job.

pllim · 2020-07-03T02:35:05Z

The RTD failure is going to bite unrelated PRs, so better get this in sooner than later.

mwcraig

This looks good, thanks for tracking down the issue! Should we open an issue as a reminder to change the version restriction when there is a new numpydoc release?

mhvk · 2020-07-04T18:18:54Z

@pllim - wow, that was good that I saw this, as I was getting quite stuck with similar warnings on my own package. What a mess! I had thought this could be solved with https://github.com/astropy/astropy/blob/master/docs/nitpick-exceptions, but that doesn't work for these errors.

Aside: for myself, I added a --keep-going flag to sphinx-build so that at least I get all errors in one go. Though it means one has to &^$&#( scroll up again to actually find the warnings.

mhvk · 2020-07-04T18:20:40Z

p.s. I now see our very own Erik pointed to this file in the answer on the stack-overflow question you pointed to... Still not sure why it doesn't seem to work here.

Cadair

All the more reason to drop numpydoc in favour of napoleon.

astrofrog · 2020-07-06T14:50:02Z

Note about the current warnings:

astropy/table/table.py:docstring of astropy.table.TableColumns.keys:: WARNING: py:class reference target not found: a set-like object providing a view on D's keys
astropy/table/table.py:docstring of astropy.table.TableColumns.values:: WARNING: py:class reference target not found: an object providing a view on D's values
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.clear:: WARNING: py:class reference target not found: None.  Remove all items from D.
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.copy:: WARNING: py:class reference target not found: a shallow copy of D
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.items:: WARNING: py:class reference target not found: a set-like object providing a view on D's items
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.keys:: WARNING: py:class reference target not found: a set-like object providing a view on D's keys
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.pop:: WARNING: py:class reference target not found: v, remove specified key and return the corresponding value.
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.update:: WARNING: py:class reference target not found: None.  Update D from dict/iterable E and F.
astropy/timeseries/__init__.py:docstring of astropy.timeseries.BoxLeastSquaresResults.values:: WARNING: py:class reference target not found: an object providing a view on D's values

This kind of issue is quite common and due to the inherited-members directive here:

.. automodapi:: astropy.timeseries
   :inherited-members:

There are three solutions:

Overload the above methods and write new docstrings
Add a skip directive to the above automodapi directive to skip the BoxLeastSquaresResults class and add a separate entry that just documents that class, without the inherited members
Figure out how to get the nitpick exceptions working for this - it's possible that the presence of a quote (') in the warnings is making this not work properly, so we need to figure out how to escape it correctly, maybe by looking at the nitpick code

astropy/wcs/__init__.py:docstring of astropy.wcs.Wcsprm.get_cdelt:: WARNING: py:class reference target not found: double array[naxis]
astropy/wcs/__init__.py:docstring of astropy.wcs.Wcsprm.get_pc:: WARNING: py:class reference target not found: double array[naxis][naxis]

This is a real error - 'double array[naxis]' is not a real type, so this should be ``numpy.ndarray` and the description should mention any dimensionality restrictions.

astropy/wcs/__init__.py:docstring of astropy.wcs.Wcsprm.get_ps:: WARNING: py:class reference target not found: list of tuples
astropy/wcs/__init__.py:docstring of astropy.wcs.Wcsprm.get_pv:: WARNING: py:class reference target not found: list of tuples

Also a real error I think - should just say list and the description should say what should be in the list?

Switching to Napoleon would not fix this - napoleon doesn't really support the full numpydoc spec, see sphinx-doc/sphinx#6861 for an example of an issue I ran into.

pllim · 2020-07-06T15:11:23Z

Thanks for the inputs, everyone!

Re: three solutions -- I did start on first two and then started doubting the reality and rolled back. I mean, numpydoc 1.1.0 is a minor release, so why are things breaking so badly for us? Why didn't it complain until now? As for the third one, I wasn't ready to mess with @embray 's magic in case this breakage will be patched upstream by numpydoc.

Re: drop numpydoc in favour of napoleon -- It's not a bed of roses. I got thousands of warnings (like 3k-4k).

So how should I proceed? Should I merge this and open a follow-up issue like @mwcraig said?

mhvk · 2020-07-06T16:13:08Z

My suggestion would be to do what you do right now, for the moment just fix the numpydoc version so tests pass again, and then raise another issue. I did get a sense over at the numpydoc issue that this might get fixed; or perhaps we try to find a fix ourselves so we can nit-pick.
Though ideally fix the "real" errors that @astrofrog pointed out. I think list of tuple may actually work, i.e., writing it as list of tuple*s* may be the problem!

pllim · 2020-07-06T16:38:52Z

What about now, @mhvk and @astrofrog ? Hope I didn't go too far with the search and replace.

mhvk

It seems good enough for now!

pllim · 2020-07-06T17:23:14Z

Thanks for the review! I'm taking the liberty to merge now so this failure won't be a blocker for other PRs, especially given it's SciPy week.

pllim · 2020-07-06T17:27:23Z

To close the loop, follow-up issue is at #10529

embray · 2020-08-03T17:39:07Z

As for the third one, I wasn't ready to mess with @embray 's magic in case this breakage will be patched upstream by numpydoc

Wait, what was my magic? I'm a little confused.

pllim · 2020-08-03T17:41:49Z

@embray , I think I meant the stuff you posted in https://stackoverflow.com/questions/11417221/sphinx-autodoc-gives-warning-pyclass-reference-target-not-found-type-warning 😅

DOC: Bump Python version on RTD build and fix failure

astrofrog · 2020-08-06T09:13:54Z

I am backporting this to v4.1.x to fix some docs failures there

pllim added Docs no-changelog-entry-needed labels Jul 2, 2020

pllim added this to the v4.2 milestone Jul 2, 2020

pllim changed the title ~~DOC: Bump Python version on RTD build~~ DOC: Bump Python version on RTD build and fix failure Jul 2, 2020

pllim added testing table timeseries labels Jul 2, 2020

This comment has been minimized.

Sign in to view

pllim force-pushed the bump-rtd-python branch from 2f427b4 to 9bd1701 Compare July 3, 2020 00:23

pllim force-pushed the bump-rtd-python branch from 9bd1701 to 4963cee Compare July 3, 2020 00:46

pllim mentioned this pull request Jul 3, 2020

Replace numpydoc with napoleon astropy/sphinx-astropy#33

Closed

pllim force-pushed the bump-rtd-python branch 2 times, most recently from 913ad89 to 2960c82 Compare July 3, 2020 02:09

pllim added 🔥 Critical and removed table timeseries labels Jul 3, 2020

pllim requested a review from astrofrog July 3, 2020 02:34

pllim added the Ready-for-final-review label Jul 3, 2020

mwcraig approved these changes Jul 3, 2020

View reviewed changes

Cadair approved these changes Jul 6, 2020

View reviewed changes

pllim mentioned this pull request Jul 6, 2020

Add option to detect either of col_starts or col_ends for fixed_width columns from header #10526

Closed

pllim added 3 commits July 6, 2020 12:17

Bump Python version on RTD build [ci skip]

e4688ec

DOC: Pin numpydoc for now [ci skip]

0dafd81

DOC: Fix double array in WCS docstrings [ci skip]

62d6c1c

pllim force-pushed the bump-rtd-python branch from 2960c82 to 62d6c1c Compare July 6, 2020 16:32

DOC: Fix list of tuples in WCS docstrings [ci skip]

0c04058

mhvk mentioned this pull request Jul 6, 2020

Sphinx emits "WARNING: py:class reference target not found" with numpydoc 1.1.0 numpy/numpydoc#275

Closed

mhvk approved these changes Jul 6, 2020

View reviewed changes

pllim merged commit 3a8732f into astropy:master Jul 6, 2020

pllim deleted the bump-rtd-python branch July 6, 2020 17:23

pllim mentioned this pull request Jul 6, 2020

MNT: Undo numpydoc pin when we can #10529

Closed

pllim mentioned this pull request Jul 6, 2020

DOC: Unpin numpydoc, add nitpick exception rules #10530

Merged

astrofrog pushed a commit that referenced this pull request Aug 6, 2020

Merge pull request #10524 from pllim/bump-rtd-python

215ea55

DOC: Bump Python version on RTD build and fix failure

astrofrog modified the milestones: v4.2, v4.1 Aug 6, 2020

Conversation

pllim commented Jul 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

This comment has been minimized.

pllim commented Jul 2, 2020

Uh oh!

This comment has been minimized.

pllim commented Jul 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pllim commented Jul 3, 2020

Uh oh!

pllim commented Jul 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pllim commented Jul 3, 2020

Uh oh!

mwcraig left a comment

Choose a reason for hiding this comment

Uh oh!

mhvk commented Jul 4, 2020

Uh oh!

mhvk commented Jul 4, 2020

Uh oh!

Cadair left a comment

Choose a reason for hiding this comment

Uh oh!

astrofrog commented Jul 6, 2020

Uh oh!

pllim commented Jul 6, 2020

Uh oh!

mhvk commented Jul 6, 2020

Uh oh!

pllim commented Jul 6, 2020

Uh oh!

mhvk left a comment

Choose a reason for hiding this comment

Uh oh!

pllim commented Jul 6, 2020

Uh oh!

pllim commented Jul 6, 2020

Uh oh!

embray commented Aug 3, 2020

Uh oh!

pllim commented Aug 3, 2020

Uh oh!

astrofrog commented Aug 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

pllim commented Jul 2, 2020 •

edited

Loading

pllim commented Jul 3, 2020 •

edited

Loading

pllim commented Jul 3, 2020 •

edited

Loading