Skip to content

ENH: Make bincount output same type as weights, fixes #6854 - #7464

Closed
jaimefrio wants to merge 1 commit into
numpy:masterfrom
jaimefrio:bincount_dispatch
Closed

jaimefrio wants to merge 1 commit into
numpy:masterfrom
jaimefrio:bincount_dispatch

Conversation

@jaimefrio

Copy link
Copy Markdown
Member

This PR adds a bincount_dispatch function to arraytypes.c.src,
that can perform a weighted bincount for most of the native types,
and uses it in arr_bincount. This is notably different from the
current behavior, that always casts weights to npy_double. The new
behavior is:

  • The output is always of the same type as weights (but in native
    byte order). There is no type promotion, like in np.sum, so
    overflow in the output is very likely for small integer types.
  • Boolean arrays are OR'ed, rather than summed, to avoid weird
    overflow behaviors.
  • Works for complex arrays, probably the main use case.
  • Has no fallback for non-native types. Perhaps we should still
    attempt to cast the input to double if the first call fails?

I have also added the possibility of broadcasting weights, i.e. if
it is a scalar or a single item array, it will work with any size
list.

And contiguous arrays are no longer required, everything can work
without a copy if the arrays are aligned and in native byte order.

@jaimefrio

Copy link
Copy Markdown
Member Author

Documentation changes are missing, will leave those out until we have fully defined what the behavior should be.

This PR adds a bincount_dispatch function to arraytypes.c.src,
that can perform a weighted bincount for most of the native types,
and uses it in arr_bincount. This is notably different from the
current behavior, that always casts weights to npy_double. The new
behavior is:

 * The output is of the same type as weights (but in native byte
   order), but (unsigned) integer types and bools are promoted to
   at least a (unsigned) long, the same type promotion in np.su.
 * Works for complex arrays, probably the main use case.
 * Has no fallback for non-native types. Perhaps we should still
   attempt to cast the input to double if the first call fails?

I have also added the possibility of broadcasting weights, i.e. if
it is a scalar or a single item array, it will work with any size
list.

And contiguous arrays are no longer required, everything can work
without a copy if the arrays are aligned and in native byte order.
@jaimefrio

Copy link
Copy Markdown
Member Author

As per the mailing list discussion, I have now added integer type promotion to at least a C long.

npy_half_to_float(*w));
#elif @iscomplex@
o[0] += w[0];
o[1] += w[1];

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this correct for npy_bool? Or should we be doing *o += (*w != 0)?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be correct pretty much always, but most of the ufunc code guards it, so I would say guard it here as well. If you view an array as bool, I am not sure this might not break, if you astype it, I think it probably will but....

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't read it really, but I would prefer some extra test with strided weights though not sure how settled the overall discussion was.

@ghost

ghost commented Apr 23, 2016

Copy link
Copy Markdown

Maybe there could be a dtype argument, for easy backward compatibility with the old cast-everything-to-double behavior? Though I'm not sure what that would do in the absence of weights...

@homu

homu commented Oct 22, 2016

Copy link
Copy Markdown
Contributor

☔ The latest upstream changes (presumably #8109) made this pull request unmergeable. Please resolve the merge conflicts.

@charris

charris commented Oct 22, 2016

Copy link
Copy Markdown
Member

Should be pretty easy to rebase.

@WarrenWeckesser

Copy link
Copy Markdown
Member

@jaimefrio, it appears that you have abandoned this pull request. I'm closing it now, but don't hesitate to reopen it if you resume working on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants