Should np.ma.masked be hashable? #4660

Activity

1. jjhelmus commented on Oct 21, 2015

   

   jjhelmus

   on Oct 21, 2015

   Contributor

   More actions

   np.ma.masked is an instance of the np.ma.MaskedArray class and therefore is not hashable in Python 3 for the same reason why all instances of np.ma.MaskedArray or even np.ndarray are not hashable, their contents can change over time.

   # Python 3.4, NumPy 1.9.3
   import numpy as np
   a = np.ndarray(1)
   b = np.ma.MaskedArray()
   hash(a)
   # TypeError: unhashable type: 'numpy.ndarray'
   hash(b)
   # TypeError: unhashable type: 'MaskedArray'
   hash(np.ma.masked)
   # TypeError: unhashable type: 'MaskedConstant'

   How can you change the contents of np.ma.masked? Easy, just modify the _data or _mask attributes. But please don't actually do this, bad things may result.

   print(np.ma.masked._data)
   # 0.0
   np.ma.masked._data[()] = 1
   print(np.ma.masked._data)
   # 1.0
2. ahaldane commented on Oct 22, 2015

   

   ahaldane

   on Oct 22, 2015

   Member

   More actions

   The example a = {np.ma.masked: 'AAA'} shouldn't work in python2 either, right? It doesn't work on my machine with 1.10.
3. jjhelmus commented on Oct 22, 2015

   

   jjhelmus

   on Oct 22, 2015

   Contributor

   More actions

   @ahaldane Correct, a = {np.ma.masked: 'AAA'} should not work in either Python 2 or 3 since np.ma.masked is mutable.

   From a git bisect is looks like PR #5326 fixed the old broken behavior where np.ma.masked was hashable and allowed to be a dictionary key. After this fix the example will raise a TypeError which I think is the correct behavior.

   This behavior is not covered by any unit tests, it may be prudent to add tests along the lines of:

   import collections
   
   def test_ma_collections_hashable():
       x = np.ma.MaskedArray([])
       assert not isinstance(x, collections.Hashable)
   
       y = np.ma.masked
       assert not isinstance(y, collections.Hashable)

   Such tests might currently fail in Windows due to issue #5647.
4. taldcroft commented on Jun 4, 2017

   

   taldcroft

   on Jun 4, 2017

   Last edited by taldcroft

   More actions

   Coming back to this because of astropy/astropy#6153 (comment), I still don't understand why np.ma.masked should not be hashable. It is an instance of MaskedConstant, which strongly implies that it is not mutable (even if it earlier derives from MaskedArray). What is the point of naming something a constant if it can be changed?

   Indeed if you try to change it via public methods an exception is raised. (I'm not concerned that there is a back-door private attribute, if somebody messes with np.ma.masked in that way then they get what they deserve).

   In [7]: np.ma.masked[()] = 10
   ---------------------------------------------------------------------------
   MaskError                                 Traceback (most recent call last)
   <ipython-input-7-a4d285aabd96> in <module>()
   ----> 1 np.ma.masked[()] = 10
   
   /Users/aldcroft/anaconda3/envs/astropy/lib/python3.5/site-packages/numpy/ma/core.py in __setitem__(self, indx, value)
      3227         """
      3228         if self is masked:
   -> 3229             raise MaskError('Cannot alter the masked element.')
      3230         _data = self._data
      3231         _mask = self._mask
   
   MaskError: Cannot alter the masked element.
   
5. eric-wieser commented on Sep 28, 2017

   

   eric-wieser

   on Sep 28, 2017

   Last edited by eric-wieser

   Member

   More actions

   Easy, just modify the _data or _mask attributes.

   This no longer works

   I still don't understand why np.ma.masked should not be hashable

   I can see one argument: it represents an unknown value, and an unknown value has an unknown hash.
6. eric-wieser

   added

   component: numpy.mamasked arraysmasked arrays

   on Sep 28, 2017
7. mhvk commented on Sep 28, 2017

   

   mhvk

   on Sep 28, 2017

   Contributor

   More actions

   I can see one argument: it represents an unknown value, and an unknown value has an unknown hash.

   but np.nan can be used as a key. Since we often write <something> is masked, I think it is reasonable for it to be assumed constant and thus hashable.
8. eric-wieser

   added

   54 - Needs decision

   on Dec 12, 2017
9. endolith commented on Aug 3, 2019

   

   endolith

   on Aug 3, 2019

   Last edited by endolith

   Contributor

   More actions

   Somewhat on-topic: Would it make sense that when an array's flag.writable is set to False, that the array becomes hashable, since it can no longer be modified? (My immediate goal is just to memoize functions that take numpy arrays as inputs.)
10. eric-wieser commented on Aug 4, 2019

    

    eric-wieser

    on Aug 4, 2019

    Member

    More actions

    That's an interesting idea, but unfortunately doesn't work too well - I can create a read-only view of a writeable array, and the hash will vary with the source array