Repository navigation
'fix_inputs' crashes when reshaping output #11360
Description
Description
I am trying to simulate a situation when fix_inputs is applied to a slightly convoluted model (inputs-to-outputs is remapped). Created model works for simple scalar inputs but fails for numpy.ndarray data.
Steps to Reproduce
In [1]: from astropy.modeling import models
...: import numpy as np
...: m = models.Identity(3) | models.Mapping((2, 1, 0))
...: m.bounding_box = ((0, 100), (0, 200), (0, 50))
...: mf = models.fix_inputs(m, {2: 22})
...: t = mf | models.Mapping((2, 1), n_inputs=3)
...: print(mf(1, 2)) # works: (22, 2, 1)
...: print(t(1, 2)) # works: (1, 2)
...: print(t(np.ones((3, 5)), np.ones((3, 5)))) # fails
(22.0, 2.0, 1.0)
(1.0, 2.0)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-1-71292af2df6f> in <module>
7 print(mf(1, 2)) # works: (22, 2, 1)
8 print(t(1, 2)) # works: (1, 2)
----> 9 print(t(np.ones((3, 5)), np.ones((3, 5)))) # fails
.../lib/python3.8/site-packages/astropy/modeling/core.py in __call__(self, *args, **kw)
2818 return outputs
2819 else:
-> 2820 return self._evaluate(*args, **kw)
2821
2822 def _evaluate(self, *args, **kw):
.../lib/python3.8/site-packages/astropy/modeling/core.py in _evaluate(self, *args, **kw)
2824 if op != 'fix_inputs':
2825 if op != '&':
-> 2826 leftval = self.left(*args, **kw)
2827 if op != '|':
2828 rightval = self.right(*args, **kw)
.../lib/python3.8/site-packages/astropy/modeling/core.py in __call__(self, *args, **kw)
2818 return outputs
2819 else:
-> 2820 return self._evaluate(*args, **kw)
2821
2822 def _evaluate(self, *args, **kw):
.../lib/python3.8/site-packages/astropy/modeling/core.py in _evaluate(self, *args, **kw)
2882 for ind, val in subargs:
2883 newargs.insert(ind, val)
-> 2884 return self.left(*newargs, **kw)
2885
2886 @property
.../lib/python3.8/site-packages/astropy/modeling/core.py in __call__(self, *args, **kw)
2818 return outputs
2819 else:
-> 2820 return self._evaluate(*args, **kw)
2821
2822 def _evaluate(self, *args, **kw):
.../lib/python3.8/site-packages/astropy/modeling/core.py in _evaluate(self, *args, **kw)
2839 elif op == '|':
2840 if isinstance(leftval, tuple):
-> 2841 return self.right(*leftval, **kw)
2842 else:
2843 return self.right(leftval, **kw)
.../lib/python3.8/site-packages/astropy/modeling/core.py in __call__(self, *args, **kwargs)
917 new_args, kwargs = self._get_renamed_inputs_as_positional(*args, **kwargs)
918
--> 919 return generic_call(self, *new_args, **kwargs)
920
921 def _get_renamed_inputs_as_positional(self, *args, **kwargs):
.../lib/python3.8/site-packages/astropy/modeling/core.py in generic_call(self, *inputs, **kwargs)
4123 if self.n_outputs == 1:
4124 outputs = (outputs,)
-> 4125 outputs = self.prepare_outputs(format_info, *outputs, **kwargs)
4126
4127 outputs = self._process_output_units(inputs, outputs)
.../lib/python3.8/site-packages/astropy/modeling/core.py in prepare_outputs(self, format_info, *outputs, **kwargs)
1779
1780 if len(self) == 1:
-> 1781 return _prepare_outputs_single_model(outputs, format_info)
1782 else:
1783 return _prepare_outputs_model_set(self, outputs, format_info, model_set_axis)
.../lib/python3.8/site-packages/astropy/modeling/core.py in _prepare_outputs_single_model(outputs, format_info)
3903 outputs[idx] = output.item()
3904 else:
-> 3905 outputs[idx] = output.reshape(broadcast_shape)
3906
3907 return tuple(outputs)
ValueError: cannot reshape array of size 1 into shape (3,5)I do not think this code should be failing: it produces (maps) correct inputs to correct outputs for scalar inputs but fails for array inputs.
System Details
macOS-10.15.7-x86_64-i386-64bit
Python 3.8.5 (default, Sep 4 2020, 02:22:02)
[Clang 10.0.0 ]
Numpy 1.20.1
astropy 4.2
Scipy 1.6.0
Matplotlib 3.3.4
Activity
mcara commentedon Mar 2, 2021
CC: @perrygreenfield
In addition, this works:
perrygreenfield commentedon Mar 4, 2021
Did you figure this out? I think I have an idea of what went wrong if you didn't.
mcara commentedon Mar 4, 2021
No, I did not investigate this further. I just implemented a workaround. Is it a user error (again)?
perrygreenfield commentedon Mar 4, 2021
I don't think so. But let me see if I can track it down.
perrygreenfield commentedon Mar 4, 2021
I'm presuming that the fix_inputs value is not getting the standard treatment for input values that permit scalars to be broadcast with other inputs. But my simple workaround cases with single values haven't worked either. There is a lot of preparation of input values and handling of output values to deal with shape issues. I'm guessing the value supplied by fix_inputs is not getting a consistent handling. Just a wild guess.
perrygreenfield commentedon Mar 4, 2021
My more refined guess starts with an explanation of what goes on before a model is evaluated. Early on in a call to the model, the list of input values is analyzed for consistency of the input shapes, and the required broadcasted shape is saved for use with the outputs. When the evaluation produces the outputs, the post evaluation code attempts to broadcast each output to the expected consistent shape analyzed from the inputs. The inputs it gets in your case are consistent with (3, 5). The input preprocessing also converts all scalars to array scalars.
The above is irrelevant. I just like going on a wild goose chase.
Try this
And compare to what you expected to get.
perrygreenfield commentedon Mar 4, 2021
I spent some time trying to understand the broadcasting processing and couldn't figure the logic of it. Until it struck me that it made assumptions inconsistent with Mapping. Then I tried the above and that confirmed it. The problem is not with fix_inputs, it is with modeling. So there! (And it was pre-existing code)
mcara commentedon Mar 4, 2021
Note [to the reader]:
m = astropy.modeling.models.Mapping((0, 1))works fine but mapping(1, 0)fails withperrygreenfield commentedon Mar 4, 2021
Correct. That is because it is assuming that the outputs will have shape () and and (3,)
nden commentedon Mar 28, 2021
This is essentially the same issue asa #10170 and #9953
pllim commentedon Mar 29, 2021
Should we close the duplicate issues? There can be only one.
Possible fixes for issue astropy#11360. More investigation is necessary.
Added tests for astropy#11360 fixes.
Potential fix for issue astropy#10170