Incompatible with sphinx>4 (submodules of submodules)

@larrybradley

Not sure what is happening but we need to make sure it is not a bug here before reporting it upstream to sphinx. This is blocking astropy/astropy#11763 .

@larrybradley narrowed down the issue in astropy/astropy#11723 (comment)

I think this is a real issue. I'm getting this same error in photutils. It appears to be due to submodules of submodules:

mod1
    __init__.py  (from mod2 import *)
    mod2
       __init__.py (from a import *)
       a.py (defines myfunc, and __all__ = ['myfunc'])

In this case, Sphinx is now defining duplicate references as:
package.mod1.myfunc and package.mod1.mod2.myfunc and raising the
"duplicate object descriptions".

This started with Sphinx 4 release. Not sure if this was an unintended bug there or if something needs to be fixed in astropy-sphinx for compatiblity with Sphinx 4.

I'll have a look at this the coming week ✋🏼

🙏 Thank you! 🙏

xref: sphinx-doc/sphinx#9121 and sphinx-doc/sphinx#9128

@pllim

To clarify, this patch was released in v4.0.0b2, and it's present in all versions since then. @pllim I misled you into thinking that our bug #130 would be fixed by this, but it's the opposite: it's caused by it.

In the case of photutils, these are the affected classes: photutils.psf.matching.windows.CosineBellWindow, photutils.psf.matching.windows.HanningWindow, photutils.psf.matching.windows.SplitCosineBellWindow, photutils.psf.matching.windows.TopHatWindow, photutils.psf.matching.windows.TukeyWindow.

It's interesting that photutils.psf.matching.fourier functions do not seem to be affected. I added a photutils.psf.matching.fourier.TESTCLASS class, and this also triggered a warning. Not sure why classes and functions work differently on this.

Will add more information soon.

@astrojuanlu

@astrojuanlu , any update from the Sphinx side? If not, we might implement a workaround so we can remove the pin.

Sorry I didn't have time to have a deeper look at this. I can do the week of August 9th, feel free to proceed with a workaround if you're in a hurry.

No hurry (yet). Thanks for the update!

See some analysis here astropy/photutils#1238

I have a minimal sphinx project producing the issue here:

https://github.com/maxnoe/sphinx-automodapi-duplicated-warning

@eteq

@eteq said he was gonna look at this... any update, Erik? Thanks!

If nothing is happening here, see astropy/photutils#1306 for downstream fix.

Related astropy PRs: astropy/astropy#12270 and astropy/astropy#13015

So it seems we should somehow generate the autodoc templates using :canonical: for the classes etc. that are duplicated:
https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#directive-option-py-class-canonical
(But I have no idea how to do that)

This needs to go into the output of sphinx-automodapi somehow, which in turn seems to rely on sphinx-autodoc. Which doesn't seem to forward options to the class directive.

E.g. in ctapipe/docs/api I have three files for the same class cause it is imported in two __init__.py:

ctapipe.calib.camera.calibrator.CameraCalibrator.rst
ctapipe.calib.camera.CameraCalibrator.rst
ctapipe.calib.CameraCalibrator.rst

These are the files autogenerated by automod-api and contain rst instructions using sphinx.ext.autodoc: https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#directive-autoclass:

CameraCalibrator
================

.. currentmodule:: ctapipe.calib.camera

.. autoclass:: CameraCalibrator
   :show-inheritance:

   .. rubric:: Attributes Summary

   .. autosummary::

      ~CameraCalibrator.apply_peak_time_shift
      ~CameraCalibrator.apply_waveform_time_shift
      ~CameraCalibrator.data_volume_reducer_type
      ~CameraCalibrator.image_extractor_type

   .. rubric:: Methods Summary

   .. autosummary::

      ~CameraCalibrator.__call__

   .. rubric:: Attributes Documentation

   .. autoattribute:: apply_peak_time_shift
   .. autoattribute:: apply_waveform_time_shift
   .. autoattribute:: data_volume_reducer_type
   .. autoattribute:: image_extractor_type

   .. rubric:: Methods Documentation

   .. automethod:: __call__

So probably we should make try to get autodoc to include the :canonical: path of the class when it's run?

autodoc has apparently a way to include that:

https://github.com/sphinx-doc/sphinx/blob/b4276edd848d112b4e981011c334d27cbcb20018/sphinx/ext/autodoc/__init__.py#L1674-L1676

Not sure how to trigger it though...

Commit adding this:
sphinx-doc/sphinx@acf66bc

@maxnoe

The method you pointed @maxnoe (get_canonical_fullname) is correctly executed. But then it goes through

https://github.com/sphinx-doc/sphinx/blob/b4276edd848d112b4e981011c334d27cbcb20018/sphinx/domains/python.py#L1218-L1237

which triggers the warning.

If I take your reproduction project (very useful btw !), the generated docs seems correct. First we have a duplicate for duplicated.Foo:

.. py:class:: Foo()
   :module: duplicated
   :canonical: duplicated.foo.foo.Foo

   Bases: :py:class:`object`

   Awesome Foo class

And then another one for duplicated.foo.Foo (which is the ones triggering the warning):

.. py:class:: Foo()
   :module: duplicated.foo
   :canonical: duplicated.foo.foo.Foo

   Bases: :py:class:`object`

   Awesome Foo class

In the method I pointed above there is a dict with the duplicates:

(Pdb++) pp self.objects
{'duplicated.Foo': ObjectEntry(docname='api/duplicated.Foo', node_id='duplicated.Foo', objtype='class', aliased=False),
 'duplicated.foo.Foo': ObjectEntry(docname='api/duplicated.foo.Foo', node_id='duplicated.foo.Foo', objtype='class', aliased=False),
 'duplicated.foo.foo.Foo': ObjectEntry(docname='api/duplicated.Foo', node_id='duplicated.Foo', objtype='class', aliased=True)}

So I'm thinking that this may actually be a bug in Sphinx which correctly handles the case of one duplicate but not when there are more than one ?

See sphinx-doc/sphinx#10348 (comment) for a very helpful and detailed description of the problem and the different options. Maybe we could add an option in automodapi to add :noindex: when necessary ?

Add option to set :noindex:

Reading the comment, I think a good solution (however I don't know how easy it would be) would be 2):

Enable sphinx_automodapi to add the :noindex: option sometimes. This way, you can still use the extension but the newly documented objects are not added to the index, thus not exactly cross-referenceable and would not trigger any warning.

What do you think about the feasibility of adding a :no-index: option to .. automod-api that is a list of objects for which :no-index should be added?

E.g. for the minimal example you'd put:

.. automodapi:: duplicated.foo
   :no-index: duplicated.foo.Foo, duplicated.foo.Bar

?

Add option to set :noindex: (#150)

Navigation Menu

Repository navigation

Incompatible with sphinx>4 (submodules of submodules) #130

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Incompatible with sphinx>4 (submodules of submodules) #130

Description

Activity

astrojuanlu commented on May 22, 2021

pllim commented on May 22, 2021

astrojuanlu commented on May 24, 2021

pllim commented on May 24, 2021

astrojuanlu commented on May 24, 2021

pllim commented on Jul 26, 2021

astrojuanlu commented on Jul 27, 2021

pllim commented on Jul 28, 2021

astrojuanlu commented on Sep 7, 2021

maxnoe commented on Mar 29, 2022

pllim commented on Mar 29, 2022

saimn commented on Apr 5, 2022

maxnoe commented on Apr 5, 2022

maxnoe commented on Apr 5, 2022

saimn commented on Apr 11, 2022

saimn commented on Apr 19, 2022

maxnoe commented on Jun 13, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions