Skip to content

ENH: Augment ufunc.reduce with state #8773

Open

Description

@eric-wieser

Right now, reduce can be roughly described as:

it = iter_along_axis(arr)  #pseudocode
result = func.identity or next(it)
for i in it:
    result = func(result, i)
return result

Or diagramatically:

       x2     x3     x3      xn
       |      |      |       |
       V      V      V       V
x1 -> (f) -> (f) -> (f) ... (f) --> result

What I'm proposing is a more powerful variant that allows extra state to be preserved between each function call:

it = iter_along_axis(arr)  #pseudocode
result =  func.identity or next(it)
state = None
for i in it:
    result, state = func(result, i, reduce_state=state)
return result
         x2     x3     x3      xn
         |      |      |       |
         V      V      V       V
x1   -> ( ) -> ( ) -> ( ) ... ( ) --> result
        (f)    (f)    (f)     (f)
None -> ( ) -> ( ) -> ( ) ... ( )

This makes ufuncs capable of describing:

Activity

  1. eric-wieser commented on Mar 12, 2017

    @eric-wieser
    MemberAuthor

    An alternative way to implement this might be to just allow the ufunc to modify its reduce_state parameter directly, like an out argument

  2. njsmith commented on Mar 12, 2017

    @njsmith
    Member
  3. eric-wieser commented on Mar 12, 2017

    @eric-wieser
    MemberAuthor

    @njsmith : I was wrong about gcd - ufunc.reduce is already able to handle this.

    Yes, gufuncs do provide a more general solution, but they don't support an axis= argument for the basic case, nor can they be accumulated.

  4. mhvk commented on Oct 21, 2017

    @mhvk
    Contributor

    This makes me realize I should get back to #8819, implementing axes for gufuncs...

  5. hameerabbasi commented on Feb 27, 2018

    @hameerabbasi
    Contributor

    This would also make things like all_equal relatively simple without an intermediate array. For example, the state could be True by default and then f would do state = state && (x_i == x_{i-1}.

    You would have to output the state in this case though.

  6. mhvk commented on Feb 28, 2018

    @mhvk
    Contributor

    Looking at this again, the logical path would seem to be a gufunc, with reduce just being a special case of one. #8819 is about to provide the ability to work along a single axis. Combined axes or axes=None is less trivial, though (might would need a transpose or ravel preprocessing step).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions