ENH: Make inspect.signature work for numpy C functions

Which involves including --\n\n in the docstrings. See http://stackoverflow.com/a/41245451/102441

I'm currently working on a POC, not sure if it's possible the way numpy does it right now. I'm not sure at which stage the __text_signature__ thing intercepts the documentation parsing.

numpydoc convention is to put the signature first, without the --.

@pv

@pv The signature lines (including the -- after the signature) are removed from the "docstring" in this process. Numpydoc can (at least since numpy/numpydoc#73) correctly recognize that signature as well.

Frustratingly, python can't deal with a signature like func(dtype=float), due to these lines

I see, this is something supported by Python function objects or types.
That's fine then as far as numpydoc is concerned.

Small patch to function_base.py of

--- a/numpy/lib/function_base.py
+++ b/numpy/lib/function_base.py
@@ -4511,7 +4511,7 @@ def trapz(y, x=None, dx=1.0, axis=-1):
 
 
 #always succeed
-def add_newdoc(place, obj, doc):
+def add_newdoc(place, obj, doc, sig=None):
     """
     Adds documentation to obj which is in module place.
 
@@ -4532,15 +4532,23 @@ def add_newdoc(place, obj, doc):
     routine never raises an error the caller must check manually
     that the docstrings were changed.
     """
+    def _add_doc(obj, sig_or_doc, doc=None):
+        if doc is not None:
+            doc = '    ' + sig_or_doc+ '\n--\n\n' + doc
+        else:
+            doc = sig_or_doc
+        doc = doc.strip()
+        add_docstring(obj, doc)
+
     try:
         new = getattr(__import__(place, globals(), {}, [obj]), obj)
         if isinstance(doc, str):
-            add_docstring(new, doc.strip())
+            _add_doc(new, doc, sig)
         elif isinstance(doc, tuple):
-            add_docstring(getattr(new, doc[0]), doc[1].strip())
+            _add_doc(getattr(new, doc[0]), *doc[1:])
         elif isinstance(doc, list):
             for val in doc:
-                add_docstring(getattr(new, val[0]), val[1].strip())
+                _add_doc(getattr(new, val[0]), *val[1:])
     except:
         pass

a regex replacing """\n ([\w.]+\(.*\))\n\n with "\1",\n """\n in add_newdocs.py, and a couple fixes of single/double quotes does the trick for populating __text_signature__.

Unfortunately, there are a lot of cases where this causes help(func) to throw a RuntimeError when it fails to parse a somewhat reasonable signature. Perhaps there's a patch for upstream python there

@eric-wieser

@eric-wieser That diff is a really nice way to do it. That would also allow to not add the \n--\n\n in Python < 3.4.

My proof of concept (https://github.com/MSeifert04/numpy/tree/signature_thing) was more manually, I just modified the ndarray.all docstring to see if it worked:

(test) C:->pip install git+https://github.com/MSeifert04/numpy@signature_thing
Collecting git+https://github.com/MSeifert04/numpy@signature_thing
  Cloning https://github.com/MSeifert04/numpy (to signature_thing) to c:\-
Installing collected packages: numpy
  Found existing installation: numpy 1.13.0.dev0+b453d48
    Uninstalling numpy-1.13.0.dev0+b453d48:
      Successfully uninstalled numpy-1.13.0.dev0+b453d48
  Running setup.py install for numpy ... done
Successfully installed numpy-1.13.0.dev0+3e5c0dc

(test) C:->python
Python 3.5.3 |Continuum Analytics, Inc.| (default, Feb 22 2017, 21:28:42) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> from inspect import signature
>>> signature(np.ndarray.all)
<Signature (axis=None, out=None, keepdims=False)>
>>> exit()

Regarding the dtype RuntimeError: Just use dtype='float' in the signature?

I think having incorrect signatures like 'float' is worse than none at all. We could just check if the signature parses correctly before assigning it

Why is "float" incorrect?

Oops, you're right. I wasn't aware that numpy did string conversion of dtypes like that. Either way, I don't think we want to encourage passing 'float' over float, which is what the effect would be

@eric-wieser

@eric-wieser I may be wrong but float isn't correct either because the correct default is np.float_?

@MSeifert

@MSeifert: you're right, that would be best of all.

I wonder if the array_function wrappers have made this change less worthwhile...

It seems like you can also add a __signature__ property on the function to actually provide the Signature instance?

For example, signature() uses this to stop unwrapping if any object in the chain has a __signature__ attribute defined.

This might give a bit more flexibility than the __text_signature__.

I don't believe it's possible to attach extra properties to C extension methods, although I remember seeing a PEP about it.

Navigation Menu

Repository navigation

ENH: Make inspect.signature work for numpy C functions #8734

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

ENH: Make inspect.signature work for numpy C functions #8734

Description

Activity

MSeifert04 commented on Mar 3, 2017

pv commented on Mar 3, 2017

MSeifert04 commented on Mar 3, 2017

eric-wieser commented on Mar 3, 2017

pv commented on Mar 3, 2017

eric-wieser commented on Mar 3, 2017

MSeifert04 commented on Mar 3, 2017

eric-wieser commented on Mar 3, 2017

MSeifert04 commented on Mar 3, 2017

eric-wieser commented on Mar 3, 2017

MSeifert04 commented on Mar 3, 2017

eric-wieser commented on Apr 20, 2019

saulshanabrook commented on May 26, 2020

eric-wieser commented on May 26, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions