Repository navigation
Make documentation more clear on lack of support for masked quantities in tables #10119
Description
Description
The current documentation makes it seem like it should be possible to have a column with quantities that are masked. QTable for example allows for masked=True. There are many examples throughout the documentation (for example here) which show a masked column in a QTable.
Expected behavior
from astropy.table import QTable, MaskedColumn
import astropy.units as u
tbl = QTable(masked=True)
tbl.add_column(MaskedColumn([1, 2] * u.keV, mask=[True, False]))
print(tbl)
output
col0
keV
----
-
2.0Actual behavior
from astropy.table import QTable, MaskedColumn
import astropy.units as u
tbl = QTable(masked=True)
tbl.add_column(MaskedColumn([1, 2] * u.keV, mask=[True, False]))
print(tbl)
output
col0
keV
----
1.0
2.0Recommendation
Given that masked Quantities arrays are not a thing (see #1852), I would recommend that there be a note or warning adding to the documentation to make it crystal clear that it is currently not possible to make a MaskedColumn with quantities. I might even go so far as to issue a warning when a user tried to create a MaskedColumn with quantities.
System Details
import astropy; print("astropy", astropy.__version__)
'4.0.1.post1'
Activity
pllim commentedon Apr 6, 2020
cc @taldcroft and @mhvk . We discussed this in Slack and I also find the documentation confusing. I thought there was an issue already about this but I could not find it. FYI.
taldcroft commentedon Apr 6, 2020
For the record, there is #6031. This solution cannot support integer masked quantities and may have some problems in the details (as noted there, what to do with
np.sum()of a masked quantity). And there is #8133, which may be more robust but is a fair bit more tricky and hasn't yet seen a working implementation AFAIK. And there has been a philosophical difference about whether it is best to do masking with a dualQuantity / MaskedQuantityapproach or oneQuantityclass withNaNas a marker ala Pandas.I would like to get something done since since #1852 is over 6 years old and still relevant (and perfect is the enemy of good), but certainly improving documentation is better than zero.
mhvk commentedon Apr 6, 2020
I've been working on this on and off, and as, indeed, perfect is the enemy of good, I should post. It is basically a new Masked class, a bit like
Distribution, that can work on arbitray ndarray subclasses.In the meantime, though, agreed that at least the documentation should be good.
astrofrog commentedon Jun 1, 2020
@mhvk - I'd be interested in seeing the approach you've been working on! Want to open a PR so we can compare to the approaches in #6031 and #8133? 😄
taldcroft commentedon Jul 16, 2021
We have masked quantities in Tables now!!