Skip to content

MAINT: Express PyArray_DescrAlignConverter in terms of _convert_from_any - #15287

Merged
seberg merged 1 commit into
numpy:masterfrom
eric-wieser:simplify-PyArray_DescrAlignConverter
Jan 8, 2020
Merged

seberg merged 1 commit into
numpy:masterfrom
eric-wieser:simplify-PyArray_DescrAlignConverter

Conversation

@eric-wieser

@eric-wieser eric-wieser commented Jan 8, 2020 •

Copy link
Copy Markdown
Member

Builds on gh-15284, will need rebasing once that gets merged.

This also fixes an inconsequential bug, meaning the following (scary) code no longer asserts

np.typeDict['mypoint'] = [('a', 'i1'), ('b', 'i2')]
assert np.dtype('mypoint', align=True).isalignedstruct

I'm not going to legitimize this sort of thing with a test though.

@eric-wieser
eric-wieser requested a review from seberg January 8, 2020 01:02
@eric-wieser
eric-wieser force-pushed the simplify-PyArray_DescrAlignConverter branch 2 times, most recently from b06f0ad to ee0cb5d Compare January 8, 2020 15:22
@eric-wieser
eric-wieser marked this pull request as ready for review January 8, 2020 15:23
This also fixes an inconsequential bug, meaning the following (scary) code no longer asserts
```
np.typeDict['mypoint`] = [('a', 'i1'), ('b', 'i2')]
assert np.dtype('mypoint', align=True).isalignedstruct
```

I'm not going to legitimize this sort of thing with a test though.
@eric-wieser
eric-wieser force-pushed the simplify-PyArray_DescrAlignConverter branch from ee0cb5d to b36db66 Compare January 8, 2020 15:26
Comment on lines +1658 to +1662
/*
* Probably only ever dispatches to `_convert_from_type`, but who
* knows what users are injecting into `np.typeDict`.
*/
return _convert_from_any(item, align);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to keep the old behavior, we could change this to

Suggested change
/*
* Probably only ever dispatches to `_convert_from_type`, but who
* knows what users are injecting into `np.typeDict`.
*/
return _convert_from_any(item, align);
return _convert_from_any(item, 0);

@seberg

seberg commented Jan 8, 2020

Copy link
Copy Markdown
Member

Can we deprecate that typeDict nonsense? (I know, not here) And with deprecate it, I mean just freeze the content (in case we need it internally right now) unceremoniously and give an error for people attempting to do such evil things?

@eric-wieser

Copy link
Copy Markdown
Member Author

Perhaps replace it with a MappingProxy?

I think @moble might be inserting a quaternion type string in there, but maybe not. Either way, I'm not sure string definitions of new dtypes are something we want to encourage.

@seberg seberg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, much less error prone, will merge soon probably. I guess by now convert_from_any and _arraydescr_run_converter are practically identically, and convert_from_any should be removed, but I am sure you are planning that in a followup.

@seberg

seberg commented Jan 8, 2020

Copy link
Copy Markdown
Member

Hmm, you are right. I had checked rational, quaternion does use it currently. I do not like the string usage much, and would prefer if we do not use it for new stuff. One thing we could do is to add "old style" dtypes ourselves, removing the need to modify it. (OTOH, it will still break quaternions/give a warning at least).

@eric-wieser

Copy link
Copy Markdown
Member Author

At a minimum we could require that the things in that dict are dtype objects or np.scalar subclasses.

@eric-wieser

Copy link
Copy Markdown
Member Author

I guess by now convert_from_any and _arraydescr_run_converter are practically identically, and convert_from_any should be removed,

Except crucially, the former can't be trusted to set an exception, while the latter can.

@moble

moble commented Jan 8, 2020

Copy link
Copy Markdown
Contributor

I'm afraid I'm completely lost as to what's going on here, or how it relates to quaternion, but I'm entirely willing to change any code over there to work with newer approaches you guys are using. Please just let me know what I'll need to do.

@seberg

seberg commented Jan 8, 2020

Copy link
Copy Markdown
Member

Oh, I was basically saying it would be nice if we remove the hook that currently allows your users to write np.array(..., dtype="quaternion") (instead of writing np.array(..., dtype=quaternion.quaternion) (or whatever the typical name for the scalar is currently). quaternion is probably the only user of that hook, but this PR nicely highlights that it is a very brittle hook, and quite frankly, I do not see why quaternion users should use the string version.

@seberg

seberg commented Jan 8, 2020

Copy link
Copy Markdown
Member

In any case, let me merge this. Thanks Eric. Before I realized how weird exactly that fixed path was, I thought about adding a test, but nvm. that...

@seberg
seberg merged commit ee7fcb7 into numpy:master Jan 8, 2020
@seberg

seberg commented Jan 8, 2020

Copy link
Copy Markdown
Member

xref gh-15296, which is an issue for the above string registration method.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants