Repository navigation
Create ndarray (recarray?) subclass that can hold multiple Quantities of mixed units #3777
Description
Over in this comment, @mhvk suggested that if we want an array-like container for Quantities of different units, if nothing else we could just use an ndarray of dtype=object to hold multiple Quantity objects. And in principle I'm fine with that as a solution.
However, I suggested in a followup that a subclass specially for this purpose might still be useful, if nothing else, for improved representation. For example, instead of an array of Quantities looking like:
[<Quantity 1.0 m>, <Quantity 2.0 km>, <Quantity 3.0 Jy>](which looks even worse when you go > 1 dimension), it might be nice to have something that reprs/prints like
[1.0 m, 2.0 m, 3.0 Jy]etc. This would look especially nice if the repr is justified so that the columns line up :)
However, it turns out that a subclass for this purpose might be necessary even aside from aesthetic improvements, since it seems currently (at least with Numpy 1.9.1 which I've tested this on) it's not even possible to repr an array of Quantities:
In [23]: a = np.array([1 * u.km, 2 * u.J], dtype=object)
In [24]: print(a)
---------------------------------------------------------------------------
UnitsError Traceback (most recent call last)
<ipython-input-24-c5a4f3535135> in <module>()
----> 1 print(a)
C:\Python27\lib\site-packages\numpy\core\numeric.pyc in array_str(a, max_line_width, precision, suppress_small)
1713
1714 """
-> 1715 return array2string(a, max_line_width, precision, suppress_small, '
', "", str)
1716
1717 def set_string_function(f, repr=True):
C:\Python27\lib\site-packages\numpy\core\arrayprint.pyc in array2string(a, max_line_width, precision, suppress_small, separator, prefix, style, formatter)
452 else:
453 lst = _array2string(a, max_line_width, precision, suppress_small,
--> 454 separator, prefix, formatter=formatter)
455 return lst
456
C:\Python27\lib\site-packages\numpy\core\arrayprint.pyc in _array2string(a, max_line_width, precision, suppress_small, separator, prefix, formatter)
254
255 formatdict = {'bool' : _boolFormatter,
--> 256 'int' : IntegerFormat(data),
257 'float' : FloatFormat(data, precision, suppress_small),
258 'longfloat' : LongFloatFormat(precision),
C:\Python27\lib\site-packages\numpy\core\arrayprint.pyc in __init__(self, data)
639 def __init__(self, data):
640 try:
--> 641 max_str_len = max(len(str(maximum.reduce(data))),
642 len(str(minimum.reduce(data))))
643 self.format = '%' + str(max_str_len) + 'd'
C:\MinGW\msys\1.0\home\embray\src\astropy\astropy\units\quantity.pyc in __array_prepare__(self, obj, context)
293 # the unit the output from the ufunc will have.
294 if function in UFUNC_HELPERS:
--> 295 converters, result_unit = UFUNC_HELPERS[function](function,
*units)
296 else:
297 raise TypeError("Unknown ufunc {0}. Please raise issue on "
C:\MinGW\msys\1.0\home\embray\src\astropy\astropy\units\quantity_helper.pyc in helper_twoarg_comparison(f, unit1, unit2)
305
306 def helper_twoarg_comparison(f, unit1, unit2):
--> 307 converters, _ = get_converters_and_unit(f, unit1, unit2)
308 return converters, None
309
C:\MinGW\msys\1.0\home\embray\src\astropy\astropy\units\quantity_helper.pyc in get_converters_and_unit(f, *units)
283 "Can only apply '{0}' function to quantities "
284 "with compatible dimensions"
--> 285 .format(f.__name__))
286
287 return converters, units[fixed]
UnitsError: Can only apply 'less' function to quantities with compatible dimensionsIt seems that the Numpy array print code at some point tries ordering the values in the array, leading to this crash. This indicates maybe two things:
-
We could work around this specific error, I think, by making
UnitsErroraValueErrorsubclass. -
Even if we fixed that specific issue it demonstrates that an array of Quantities can't be displays as well as one might like, and it might be worth fixing that.
A further question is why would we even need an array of Quantities of different units. I think if nothing else Quantity support in modeling may end up wanting something like this.
Activity
Possible partial solution to the issue raised in astropy#3777--it doe…
Possible partial solution to the issue raised in astropy#3777--it doe…
Possible partial solution to the issue raised in astropy#3777--it doe…
mhvk commentedon Aug 11, 2016
Looking over old units issues, I saw this one. Am still somewhat intrigued, though as written I don't see how it could work, as all the mathematical operations don't make sense if different parts of a quantity have a different unit. It is similar to having an
ndarraywith mixed integer and float elements -- it just would not make much sense. Given that analogy, however, what might work is to haverecarraysubclass that has different units for its different columns (of course, this rapidly becomes aTable...). Indeed, if numpy proceeds with making it possible to define the unit on thedtype, then this would be nearly trivial to implement.mhvk commentedon Aug 11, 2016
(I removed "bug" since that aspect has been solved; it really is a "feature request" now)
[-]Create ndarray subclass that can hold multiple Quantities of mixed units[/-][+]Create ndarray (recarray?) subclass that can hold multiple Quantities of mixed units[/+]adrn commentedon Aug 11, 2016
One place where this concept would be useful is for making the
Representationclasses a bit nicer (they are a bit clunky right now IMO). If I recall, they currently store everything internally as Cartesian coordinates? But in principle, if I create aSphericalRepresentationit could avoid that conversion step unless necessary and store the position (length, angle, angle) as a single array (or just be an array/recarray subclass). Do you have any thoughts on this @mhvk or @eteq ?mhvk commentedon Aug 11, 2016
@adrn - the representation classes do store items as they are (i.e.,
SphericalRepresentationdoes store, radius, longitude, latitude). I do think there might be some benefits to usingrecarrayinstead, as it would remove some boiler-plate, but probably not a huge deal.adrn commentedon Aug 11, 2016
@mhvk oops, my mistake for mis-remembering...
embray commentedon Sep 1, 2016
Well, don't use
recarrayspecifically--that class is all but deprecated.mhvk commentedon Sep 1, 2016
I guess I meant an
ndarraywhich uses adtypewith records. It might make sense to be able to associate different units with different records.adrn commentedon Sep 18, 2017
Another place where this might be useful is when storing phase-space information, e.g., position and velocity components, so this would be a useful feature!
astrofrog commentedon Sep 19, 2017
Also for model parameters! (models have a way of accessing arrays of parameters but for models with units we just return a list for now)
mhvk commentedon Apr 12, 2021
Just to note this is in the works, as it really helps deal with ERFA ufuncs that use/return structure dtype arguments (e.g., position and velocity).
nstarman commentedon Jun 17, 2021
@mhvk, your PR closes this, right?
mhvk commentedon Jun 17, 2021
Indeed. Now ensured this will be closed once #11775 is merged.