Repository navigation
Representation arithmetic - #5301
Conversation
| recommended_units = {} # subclasses can override | ||
|
|
||
| # Ensure multiplication/division with ndarray doesn't lead to object arrays. | ||
| __array_priority__ = 50000 |
There was a problem hiding this comment.
This reminds me: Should the priority be mentioned in the docstring of the class? Or is it obvious that representations "rule'em all" (currently)?
There was a problem hiding this comment.
Really all this needs to do is to ensure any ndarray subclass has lower priority. Of course, I don't know what the highest possible one is...
|
I've left some stylistic comments. 👍 |
|
I'm trying to wrap my head around the meaning of this for spherical frames. For a unit spherical frame, is the mean the circular mean, or can one end up with points that are floating inside the unit sphere? (in which case the mean of a unit spherical frame is not a unit spherical frame). |
| if issubclass(cls, Angle): | ||
| results.append(value) | ||
| else: | ||
| result = op(value, other) |
There was a problem hiding this comment.
Does this allow for instance a spherical representation to be multiplied by a temperature? Do we need to restrict ourselves to dimensionless quantities?
There was a problem hiding this comment.
Any representation can be multiplied by any quantity: it is just the ones that carry dimensional information (i.e., not the angles, so, e.g., r and z for cylindrical) that get multiplied.
|
I'm not sure how I feel about adding to arithmetic to all different representations, since in some cases I think the meaning isn't super obvious (and there could be multiple interpretations - e.g. circular mean vs geometric mean). Just to put the idea out there, what about implementing this only in the cartesian representation class, which would then require an explicit conversion to/from cartesian? |
|
@astrofrog - originally, @StuartLittlefair and I mostly thought about operations on cartesian representations, and possible we should indeed limit ourselves to it. My main rationale for not doing so is that some operations are truly obvious in other frames as well (especially, For |
|
I'm not sure I would say |
|
Division is also not unambiguous in spherical polar coordinates - if I divide by two, will it divide the angles by two as well? I think those operations only make sense in a cartesian frame - dividing spherical coordinates by two makes no conceptual sense. |
|
For complex numbers, For representations, it seems that to do the same is the only natural option, since representations really are just different ways of representing 3D vectors. And thinking of vectors, x/|x| should be a unit vector, so that defines division pretty unambiguously too -- to me at least, it would be very weird to have longitudes or latitudes changed by division (not to mention that in that case the divisor would have to be dimensionless, so one could never get a velocity out by dividing by a quantity with time units). |
Yes sorry, that was my point - it wasn't obvious whether abs() would work like on a quantity, or if it would always give the length of the vector. If it'll always return the length of the vector, then that's fine. |
|
I guess I'm just trying to make sure that things are truly unambiguous here for users - I agree that users will think of cartesian representation as a vector, but I'm not sure if the same is true for the others (not saying I don't agree, just that it took me a while to understand how this would work for non-cartesian frames). I guess part of me would be in favor or implementing this only on the cartesian representation initially then seeing later if there is a need to extend to others. But at the end of the day, I don't have too strong an opinion, so I'll leave it up to others. I think @eteq should definitely have a say on this first, and @Cadair (who wrote the original classes with me) |
|
Agreed it is only logical to the extent one thinks of these classes all being representations of vectors. Looking forward to comments from others. p.s. I realise I forgot |
|
Looking at other possible methods, the only one left that might be useful is For this one, though, I think I would not automatically extend it to Given this, and that this really only becomes important when our minimum python version is 3.5, I think I'll leave |
|
I'm still mulling this over, but some thoughts right now: The reason for not implementing >>> cr = CartesianRepresentation(np.random.randn(3,4)*u.kpc)
>>> sr = cr.represent_as(SphericalRepresentation)
>>> cr2 = sr.to_cartesian()
>>> cr.x == cr2.x, cr.y == cr2.y, cr.z == cr2.z
(array([False, False, False, True], dtype=bool),
array([False, False, False, False], dtype=bool),
array([ True, False, True, True], dtype=bool))So that means the following happens in this PR's: Hence, it appears that equality is not commutative! Most users will be very surprised by this and not realize it until they've written buggy code. The underlying problem here is that machine precision isn't sufficient to maintain perfect round-tripping, so this holds for pretty much any pair of representations to some extent or another. Of course there are various workarounds for this, but to me (I think after being convinced by @taldcroft?), that then creates more problems than it solves. So the idea was that we should decide for the user what "equality" means beyond the standard I also echo @astrofrog's concern that most of these operations are not unambiguous for many of the representations - it might make sense to all of us in this conversation because we've thought about it a fair amount, but the vast majority of users won't, and they might make incorrect assumptions. More abstractly, it's not clear that a representation really should be thought of as "generic vector". As currently imagined they are not, rather they are specifically spatial locations. Adding all these operations starts blurring that. So I'm going to through out two alternatives that still maintain much of the capability here while not making as many assumptions:
|
|
@eteq - I'll need to think a bit more about your points, but as you already discuss
So, one alternative would be to keep (1) and (2) for all (those are also the ones that do not assume cartesian in their implementation), and move 3,4,5 to Cartesian proper (can always add them later to the others, but hard to remove). But I'll also argue for just keeping things as I proposed (except perhaps remove For coordinates, I see particular use & need once we start implementing proper motions and velocities: we will need simple ways to add and subtract vectors, divide and multiply by time spans, and calculate cross and dot products. I think methods are better there than helper functions (indeed, a good example is As for broader use, I have in fact a Now, arguably, that would be solved just as nicely with a Overall, it may boil down to a choice: I do think representations are really just different ways to represent arbitrary 3-D vectors (with arbitrary units). |
|
On |
|
I read through this thread and started off on the ambivalent side of the "err, seems a bit ambiguous, etc". But as I was reading I started thinking about the two ways we can conceptually think of representations:
From the code and math perspective I think these are equivalent [??], but if you fully adopt idea (1) then basically none of the operations really make sense (what does it mean to add two points in space?). BUT if you re-orient yourself to idea (2) then I think all of the operations make sense for all representations and you gain powerful tools. For instance, a useful equality test for two representations could be something like: So by the end I was convinced that we should accept all the operators on all representations, except for So @eteq suggested making the Now I confess that I haven't gone back through the docs to see how representations are presented, but perhaps some/many users already think of the abstract thing underneath as a vector rather than a point? The only real outstanding question to me is how to handle in general the question of type promotion for allowed operations. For instance we know that the output of most operations with |
6bd2d73 to
7752d3c
Compare
|
OK, I think there is quite clear consensus that we should not have |
|
@mhvk Sorry for the delay, but I'd like to take a close look at this over the weekend. |
taldcroft
left a comment
There was a problem hiding this comment.
I have finished another somewhat superficial review, scanning the code for things that jump out. I haven't look at the testing at all to see if is complete (@adrn?). But with those caveats I don't see any other problems with the code.
| For JPL kernels, this roughly doubles the execution time, so if one requires | ||
| only the positions, one should use ``get_body_barycentric``. [#5231] | ||
|
|
||
| - Arithmetic on representations, such as addition of two representations, |
There was a problem hiding this comment.
Suggest adding wording that this treats representation points as vectors in cartesian space. Then everything that follows is indeed "obvious".
|
|
||
| - Arithmetic on representations, such as addition of two representations, | ||
| multiplication with a ``Quantity``, or calculating the norm via ``abs``, | ||
| has now become possible. Furthermore, there are news methods ``mean``, |
| attr_classes = OrderedDict([('lon', Longitude), | ||
| ('lat', Latitude)]) | ||
| recommended_units = {'lon': u.deg, 'lat': u.deg} | ||
| _dimensional_representation = None # set to SphericalRepresentation below |
There was a problem hiding this comment.
Seems like a good use for @classproperty in utils.decorators:
@classsproperty
def _dimensional_representation():
return SphericalRepresentation
| Returns | ||
| ------- | ||
| dot_product : `~astropy.units.Quantity` | ||
| The sum of the product of the x, y, and z components of ``self`` |
There was a problem hiding this comment.
I seem to be blind to what actually is wrong here... Could you elucidate?
There was a problem hiding this comment.
Ah sorry, I was being blind and did not see the "of the product" words. So never mind.
| Returns | ||
| ------- | ||
| dot_product : `~astropy.units.Quantity` | ||
| The sum of the product of the x, y, and z components of ``self`` |
| and z components are calculated, and the result is converted back to | ||
| the same representation as the input. | ||
|
|
||
| Refer to `numpy.mean` for full documentation of the arguments. |
There was a problem hiding this comment.
This docstring and sum should still have a Returns section to document the return object.
| that frame classes may override this with their own preferred units. | ||
| """ | ||
|
|
||
| attr_classes = OrderedDict() |
There was a problem hiding this comment.
Why was this added? From looking at the code in MetaBaseRepresentation and docstring in BaseRepresentation, I think the intent is to require classes to explicitly define attr_classes (and not rely on a base class definition).
There was a problem hiding this comment.
Good question. Removing it doesn't seem to matter, so I did. I think it may be a leftover from an earlier attempt.
e6fd632 to
76b695f
Compare
|
@taldcroft - I address your comments except that for one I wasn't quite sure what you meant with a "copy and paste" error (maybe I addressed it, but please check). Also rebased, since this was getting a bit far behind. I think I'll call this ready for final review now... |
3bd3fbb to
f47e487
Compare
|
Rebased now that #5254 is in. |
|
I'm happy with this -- ok to merge 👍 |
There was a problem hiding this comment.
OK, I'm convinced this is a fine way to go, particularly given the power this approach demonstrates in #5434. I am a bit worried about this confusing users (i.e. that it's not element-wise but rather vector operations), so many of my inline comments are aimed at clearing that up... but I now agree that it's worth having this despite the confusion risk.
That said, I concur with @adrn's concern(and @taldcroft's maybe-objection?) to abs as it is implemented here. That is, to me it really seems more natural for abs to be element-wise absolute value, as that's what it means in a numpy context. That's not the same as what it is here, the 3-norm. And remember:"In the face of ambiguity, refuse the temptation to guess.". So I would advocate either of two approaches:
- Change
absto be element-wise cartesianabs(which then goes back to the self representation), and have the currentabsbe anormmethod as @adrn advocated. - Drop
abscompletely because its ambiguous (absbiguous?) and just usenorm.
I slightly favor 2, but could go either way.
I also found two other problems while experimenting - I created mhvk#20 against @mhvk's branch with the regression tests for these:
mulfails if the shape of the representation is not 1d. Some of the others do too, but I think they're all due tomulfailing? Anyway, you can see in the tests @mhvk and maybe you can tell offhand where the problem is?absdoes not give consistent types - sometimes its a Quantity and sometimes a Distance. But in either case right now, it actually yields a numpy array of object type, not a Quantity/Distance with the appropriate shape. I think the desired behavior is always a singleDistance, so that's what the test checks for.
| for component in first.components)) | ||
|
|
||
| def mean(self, *args, **kwargs): | ||
| """Return the vector mean along a given axis. |
There was a problem hiding this comment.
I'd suggest a slightly different wording here (and in sum, and in the other def mean/def sum's above and below) - "axis" could mean e.g. "x axis" or it could mean the axis keyword (which in the representation situation means subtly different things). So how about:
Returns the vector mean along each of the cartesian components. Arguments like `axis` are for the axis of *each* component.
There was a problem hiding this comment.
Again a good point. I removed "along the given axis" as I agree it is only confusing, and expanded text as follows:
"""Vector mean.
Averaging is done by converting the representation to cartesian, and
taking the mean of the x, y, and z components. The result is converted
back to the same representation as the input.
Refer to `~numpy.mean` for full documentation of the arguments, noting
that ``axis`` is the entry in the ``shape`` of the representation, and
that the ``out`` argument cannot be used.
|
|
||
| def __abs__(self): | ||
| return np.sqrt(functools.reduce( | ||
| operator.add, (getattr(self, component)**2 |
There was a problem hiding this comment.
See my narrative comment for my doubts about __abs__ in general... But one more specific item: why not just self.dot(self)**0.5? Or perhaps Distance(self.dot(self)**0.5, copy=False)? I think that will make it behave more consistently...
There was a problem hiding this comment.
I think we should avoid conversions to Distance - one might want to use CartesianRepresentations to represent velocities for example.
There was a problem hiding this comment.
self.dot is slower for representations other than Cartesian, since it first converts to Cartesian and then takes the product. Although this really only matters for Cylindrical since for the spherical ones I already override it. Still, my sense is to leave it as is.
|
|
||
| Returns | ||
| ------- | ||
| mean : `~astropy.coordinates.BaseRepresentation` |
There was a problem hiding this comment.
Here (and everywhere else below), maybe you should change `~astropy.coordinates.BaseRepresentation to same class as self or something like that? Right now it looks like it's saying the return type is always BaseRepresentation, which is of course wrong but isn't obvious to the poor user who doesn't fully understand the class heirarchy.
There was a problem hiding this comment.
Changed to just representation throughout. This means one cannot click on the link in the docs, but I agree it is much clearer (and my guess is that people usually see the help within ipython anway by typing a ?)
| return self.to_cartesian().dot(other) | ||
|
|
||
| def cross(self, other): | ||
| """Cross product of two representations. |
There was a problem hiding this comment.
Perhaps "vector cross product"? Cross product may not be ambiguous, but that makes it a bit clearer that this is the standard cross product irrespective of the current representation
| will be a subclass of `~astropy.coordinates.UnitSphericalRepresentation`. | ||
|
|
||
|
|
||
| Vector arithmetic |
There was a problem hiding this comment.
Actually I think this should go above "creating your own" - it's more likely that a user will want to know this than need to create their own...
| Vector arithmetic | ||
| ^^^^^^^^^^^^^^^^^ | ||
|
|
||
| Representations support basic vector arithmetic, in particular taking the |
There was a problem hiding this comment.
I think it's worth trying to be a bit more explicit here on what we mean by "basic vector arithmetic" - something like:
Representations support basic vector arithmetic - i.e., vector operations as applied to *cartesian* representations.
If the representation is not a `CartesianRepresentation`, the operation will be performed on the cartesian vector, but returned as the representation type of the first operand.
or similar. I realize that, strictly speaking, that's what "vector arithmetic" nominally means, but I think this subtly could be easily lost on a reader who isn't thinking carefully about this.
There was a problem hiding this comment.
I tried to do this with the note at the very end, but now agree this would come too late. I thought a bit about changing the text, but ended up deciding to move the note higher up and make it a proper .. Note::, so it stands out in the documentation.
|
@eteq - thanks for the comments. I'm really confused about the failures, though: for me, both multiplication and |
|
On |
f47e487 to
8ee76c7
Compare
Hmm, so if I add eteq/astropy@ec9cba8 (from mhvk#20) in on top of this branch as it stands right now, I get Are you seeing it pass even if you try eteq/astropy@ec9cba8, @mhvk ?
That problem, however, seems to have disappeared in the latest version, so all good there.
Ah, good point! I get it now. I can see arguments one way or another for whether it should be forced either way, but I think it's probably best to leave it alone, as it's not clear which is better (and it's all |
|
@eteq - yes, I tried your examples by hand, and they work on my machine. But just to be sure, in the latest commits I added a test (for |
8ee76c7 to
4df4d7e
Compare
|
@eteq - the new tests, including the 2-D array check, pass on travis for all settings. So, I went ahead and merged as I would like to rebase the usage to get that shipshape as well. If there still are problems with this PR, I can address them as part of #5434 (which I'll leave open until you're OK). |
|
@mhvk - hmm, I'm still seeing this error in both py2 and py3. I'll make a PR into astropy just adding the test I described and lets see what travis tells us. That way we'll at least know if its local to my computer somehow, and can try to fix the error in that PR if so. |
|
🎉 |
This aims to address #5255, by making it possible to do arithmetic on representations, such as addition of two representations, multiplication with a
Quantity, or calculating the norm viaabs. It also introduces news methodsmean,sum,dot, andcrosswith obvious meaning.cc @StuartLittlefair, @eteq, @adrn, @astrofrog