Repository navigation
ENH: vectorized histogram with axis keyword #13166
Description
I frequently wish to calculate a histogram along each dimension of an array axis. For example, if dimension 0 is time, I might want to look at how the pdf of my data is evolving in time. This would be much easier if histogram (and its multidimensional variants) accepted an axis keyword argument. Currently histogram just flattens all its data.
Reproducing code example:
import numpy as np
# create some sample data
a = np.random.randn(5, 200)
# histogram over all dimensions, not what I want
h, bins = np.histogram(a)
# loop-based version of what I would like instead
bins = np.histogram_bin_edges(a)
individual_hists = [np.histogram(a[n], bins=bins)[0][None, :] for n in range(a.shape[0])]
full_hist = np.vstack(individual_hists)
np.testing.assert_array_equal(h, full_hist.sum(axis=0))
# the syntax I would prefer to use
full_hist, bins = np.histogram(a, axis=-1)For performance reasons, it would obviously be ideal if this could be vectorized in an efficient way, rather than the naive loop I have written here.
Related stack-overflow question: https://stackoverflow.com/questions/44152436/calculate-histograms-along-axis
Activity
OmerJog commentedon Nov 11, 2019
+1
This can be useful
GregoryMorse commentedon Dec 2, 2019
I also agree that vectorized histograms is important. I have use cases where I would like to take a series of histograms along an axis. Otherwise we are stuck with for loops.
GregoryMorse commentedon Dec 2, 2019
The problem with the answer here is it neglects to use the fast method with subtraction/multiplication/comparisons/increment/decrement and bincount. Instead sorting and searching is used which is definitely going to be slower as the scale increases. histogram has a special fast route for the case of fixed number of equal sized bins. While searchsorted has no such methodology.
rabernat commentedon Dec 2, 2019
FYI, we have started a new package for this purpose called xhistogram:
It does a few things besides just the raw vectorized histogram, including integration with xarray and dask. We would welcome contributions.
GregoryMorse commentedon Dec 3, 2019
Nice that looks great, exactly my thoughts. Handles a much broader range of cases. Nice work!
[-]vectorized histogram with axis keyword[/-][+]ENH: vectorized histogram with axis keyword[/+]rossbar commentedon Mar 22, 2023
@rabernat thanks for opening, and nice work on
xhistogram. Is this still something you're interested in upstreaming or should we close this?