Skip to content

Representation arithmetic - #5301

Merged
mhvk merged 12 commits into
astropy:masterfrom
mhvk:representations-more-powerful
Oct 28, 2016
Merged

mhvk merged 12 commits into
astropy:masterfrom
mhvk:representations-more-powerful

Conversation

@mhvk

@mhvk mhvk commented Sep 2, 2016 •

Copy link
Copy Markdown
Contributor

This aims to address #5255, by making it possible to do arithmetic on representations, such as addition of two representations, multiplication with a Quantity, or calculating the norm via abs. It also introduces news methods mean, sum, dot, and cross with obvious meaning.

cc @StuartLittlefair, @eteq, @adrn, @astrofrog

recommended_units = {} # subclasses can override

# Ensure multiplication/division with ndarray doesn't lead to object arrays.
__array_priority__ = 50000

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reminds me: Should the priority be mentioned in the docstring of the class? Or is it obvious that representations "rule'em all" (currently)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really all this needs to do is to ensure any ndarray subclass has lower priority. Of course, I don't know what the highest possible one is...

@MSeifert04

Copy link
Copy Markdown
Contributor

I've left some stylistic comments. 👍

@astrofrog

Copy link
Copy Markdown
Member

I'm trying to wrap my head around the meaning of this for spherical frames. For a unit spherical frame, is the mean the circular mean, or can one end up with points that are floating inside the unit sphere? (in which case the mean of a unit spherical frame is not a unit spherical frame).

Comment thread astropy/coordinates/representation.py Outdated
if issubclass(cls, Angle):
results.append(value)
else:
result = op(value, other)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this allow for instance a spherical representation to be multiplied by a temperature? Do we need to restrict ourselves to dimensionless quantities?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any representation can be multiplied by any quantity: it is just the ones that carry dimensional information (i.e., not the angles, so, e.g., r and z for cylindrical) that get multiplied.

@astrofrog

Copy link
Copy Markdown
Member

I'm not sure how I feel about adding to arithmetic to all different representations, since in some cases I think the meaning isn't super obvious (and there could be multiple interpretations - e.g. circular mean vs geometric mean). Just to put the idea out there, what about implementing this only in the cartesian representation class, which would then require an explicit conversion to/from cartesian?

@mhvk

mhvk commented Sep 5, 2016

Copy link
Copy Markdown
Contributor Author

@astrofrog - originally, @StuartLittlefair and I mostly thought about operations on cartesian representations, and possible we should indeed limit ourselves to it. My main rationale for not doing so is that some operations are truly obvious in other frames as well (especially, abs, but also multiplication and division) and for the others the only easy-to-explain, unambiguous way of doing the arithmetic is going via cartesian, and one might as well make that the default. And once addition seems only logically done via cartesian, sum and mean follow... That said, my guess is that all but abs, __mul__ and __div__ would be rarely used for the non-cartesian representations. Furthermore, it is not completely obvious one would necessarily want the result transformed back to spherical.

For UnitSphericalRepresentation, essentially any operation on it makes the result not be on the unit sphere, so I always return a SphericalRepresentation. Obviously, a user could transform this back to the unit sphere if so desired.

@astrofrog

astrofrog commented Sep 5, 2016 •

Copy link
Copy Markdown
Member

I'm not sure I would say abs is unambiguous - if you have points in spherical polar coordinates, does it refer to the length of the vector, or does it make all values positive? (by analogy to how it works in the complex plane where it returns the length).

@astrofrog

Copy link
Copy Markdown
Member

Division is also not unambiguous in spherical polar coordinates - if I divide by two, will it divide the angles by two as well? I think those operations only make sense in a cartesian frame - dividing spherical coordinates by two makes no conceptual sense.

@mhvk

mhvk commented Sep 5, 2016

Copy link
Copy Markdown
Contributor Author

For complex numbers, abs returns the length of the vector, it does not make all values positive!

In [1]: abs(12-5j)
Out[1]: 13.0

For representations, it seems that to do the same is the only natural option, since representations really are just different ways of representing 3D vectors.

And thinking of vectors, x/|x| should be a unit vector, so that defines division pretty unambiguously too -- to me at least, it would be very weird to have longitudes or latitudes changed by division (not to mention that in that case the divisor would have to be dimensionless, so one could never get a velocity out by dividing by a quantity with time units).

@astrofrog

Copy link
Copy Markdown
Member

For complex numbers, abs returns the length of the vector, it does not make all values positive!

Yes sorry, that was my point - it wasn't obvious whether abs() would work like on a quantity, or if it would always give the length of the vector. If it'll always return the length of the vector, then that's fine.

@astrofrog

Copy link
Copy Markdown
Member

I guess I'm just trying to make sure that things are truly unambiguous here for users - I agree that users will think of cartesian representation as a vector, but I'm not sure if the same is true for the others (not saying I don't agree, just that it took me a while to understand how this would work for non-cartesian frames). I guess part of me would be in favor or implementing this only on the cartesian representation initially then seeing later if there is a need to extend to others. But at the end of the day, I don't have too strong an opinion, so I'll leave it up to others.

I think @eteq should definitely have a say on this first, and @Cadair (who wrote the original classes with me)

@mhvk

mhvk commented Sep 5, 2016

Copy link
Copy Markdown
Contributor Author

Agreed it is only logical to the extent one thinks of these classes all being representations of vectors. Looking forward to comments from others.

p.s. I realise I forgot __neg__ among the methods that can be supported. Will add it still.

@mhvk

mhvk commented Sep 5, 2016

Copy link
Copy Markdown
Contributor Author

Looking at other possible methods, the only one left that might be useful is __matmul__ (and its reverse). For CartesianRepresentation, the reverse of that would logically be matrix @ cart_rep, so __rmatmul__ could be directly mapped to self.transform(matrix). Similarly, cart_rep @ matrix would effectively map to self.transform(matrix.T).

For this one, though, I think I would not automatically extend it to SphericalRepresentation, at least as long as matrix would just be an ndarray with shape ending in 3,3, since it is not obvious that those have to be cartesian elements (unlike for addition with another representation, where one does know what form it is and can explicitly transform to cartesian).

Given this, and that this really only becomes important when our minimum python version is 3.5, I think I'll leave __matmul__ for another PR.

@eteq

eteq commented Sep 7, 2016

Copy link
Copy Markdown
Member

I'm still mulling this over, but some thoughts right now:

The reason for not implementing __eq__ in the first place was driven by the following example (or at least situations like this):

>>> cr = CartesianRepresentation(np.random.randn(3,4)*u.kpc)
>>> sr = cr.represent_as(SphericalRepresentation)
>>> cr2 = sr.to_cartesian()
>>> cr.x == cr2.x, cr.y == cr2.y, cr.z == cr2.z
(array([False, False, False,  True], dtype=bool),
 array([False, False, False, False], dtype=bool),
 array([ True, False,  True,  True], dtype=bool))

So that means the following happens in this PR's:

>>> cr = CartesianRepresentation(np.random.randn(3,4)*u.kpc)
>>> sr = cr.represent_as(SphericalRepresentation)
>>> sr == cr 
array([ True,  True,  True,  True], dtype=bool)
>>> cr == sr 
array([False, False, False, False], dtype=bool)

Hence, it appears that equality is not commutative! Most users will be very surprised by this and not realize it until they've written buggy code.

The underlying problem here is that machine precision isn't sufficient to maintain perfect round-tripping, so this holds for pretty much any pair of representations to some extent or another. Of course there are various workarounds for this, but to me (I think after being convinced by @taldcroft?), that then creates more problems than it solves. So the idea was that we should decide for the user what "equality" means beyond the standard self is other equivalence.

I also echo @astrofrog's concern that most of these operations are not unambiguous for many of the representations - it might make sense to all of us in this conversation because we've thought about it a fair amount, but the vast majority of users won't, and they might make incorrect assumptions. More abstractly, it's not clear that a representation really should be thought of as "generic vector". As currently imagined they are not, rather they are specifically spatial locations. Adding all these operations starts blurring that.

So I'm going to through out two alternatives that still maintain much of the capability here while not making as many assumptions:

  1. Create a CartesianVector that's a sub-class of cartesian representation, and it gets all the operations. It's initializer should accept representations and just pass them through if it's cartesian (so low performance overhead), otherwise it does to_cartesian. Maybe also add to_cartesian_vector or something like that to the base representation to make it easier. (I favor this one)
  2. Move all the operations to CartesianRepresentation, and make it clear in the docs that that's intentional - basically a slightly-more-convenient (but also a bit more muddled) version of option 1.

@mhvk

mhvk commented Sep 7, 2016

Copy link
Copy Markdown
Contributor Author

@eteq - I'll need to think a bit more about your points, but as you already discuss __eq__ separately, think it is good to perhaps discuss the various methods separately as well, since I think for some I can see the arguments for them being only on Cartesian much more than the others. So, in order of increasing obscurity for non-cartesian:

  1. __abs__: to me, the one that whose definition is most obvious for all representations.
  2. __mul__, __div__, __neg__, __pos__: I feel these are very well-defined, since they only make sense if they apply to the unit-ful components; and getting a normalised version out by dividing by abs() seems one of those simply useful things.
  3. dot(), cross(): clearly defined in cartesean, arguably very handy to at least allow the second element to be non-cartesian, so one can project on UnitSpherical.
  4. __add__, __sub__: these are obvious for cartesian, less so for the others. Possibly nice to subtract two spherical coordinates and get a properly directed proper motion out.
  5. mean(), sum(): follow on from __add__, really.
  6. __eq__: it was there because it was easy to do and helps writing test cases, but I agree that in practice it may be less than useful for any of them.

So, one alternative would be to keep (1) and (2) for all (those are also the ones that do not assume cartesian in their implementation), and move 3,4,5 to Cartesian proper (can always add them later to the others, but hard to remove).

But I'll also argue for just keeping things as I proposed (except perhaps remove __eq__): these are not meant for regular users -- those only see SkyCoord and I am not proposing at all that we provide arithmetic for those. These methods are meant to make future development of coordinates easier, and allow broader use of the representations as a vector.

For coordinates, I see particular use & need once we start implementing proper motions and velocities: we will need simple ways to add and subtract vectors, divide and multiply by time spans, and calculate cross and dot products. I think methods are better there than helper functions (indeed, a good example is transform_to, which used to be a helper function). Also, I do think at that point we want to be able to use the various representations to represent velocity vectors.

As for broader use, I have in fact a QuantityVector class PR sitting in my git locally, which has the benefit of allowing arbitrary dimension, but also made me realise that this is not a logical Quantity subclass, as too many methods and ufuncs just don't make sense on vectors. This PR solves that conundrum for me.

Now, arguably, that would be solved just as nicely with a CartesianVector class, but then why prevent me from transforming it to spherical coordinates if I like that?

Overall, it may boil down to a choice: I do think representations are really just different ways to represent arbitrary 3-D vectors (with arbitrary units).

@mhvk

mhvk commented Sep 7, 2016

Copy link
Copy Markdown
Contributor Author

On __eq__, one reason for having it is that it would be nice if rep[0] == rep[0] would yield True (and similarly for other slices, getting their overlap, etc.). But your argument is strong too... One could simply not convert to any other representation. Or just not implement it. Anyway, not the most important of the things to decide!

@taldcroft

taldcroft commented Sep 7, 2016 •

Copy link
Copy Markdown
Member

I read through this thread and started off on the ambivalent side of the "err, seems a bit ambiguous, etc". But as I was reading I started thinking about the two ways we can conceptually think of representations:

  1. Representation of an abstract point in a frame
  2. Representation of an abstract vector in a frame from the origin to the point

From the code and math perspective I think these are equivalent [??], but if you fully adopt idea (1) then basically none of the operations really make sense (what does it mean to add two points in space?). BUT if you re-orient yourself to idea (2) then I think all of the operations make sense for all representations and you gain powerful tools. For instance, a useful equality test for two representations could be something like:

np.all(abs(r1 - r2) < 1e-7) 

So by the end I was convinced that we should accept all the operators on all representations, except for __eq__, which is basically not going to work.

So @eteq suggested making the Vector bit explicit with a subclass, but I think that at the end of the day this would just end up creating confusion and duplication where it really isn't necessary. I.e. users would ask "when should I use CartesianRepresentation and when should I use CartesianVector?" The answer would be "use the vector if you might want to use this list of operators on it later". But there is nothing code wise that prevents doing this all in the *Representation classes, it is just conceptual.

Now I confess that I haven't gone back through the docs to see how representations are presented, but perhaps some/many users already think of the abstract thing underneath as a vector rather than a point?

The only real outstanding question to me is how to handle in general the question of type promotion for allowed operations. For instance we know that the output of most operations with UnitSpherical needs to be promoted to a representation that isn't Unit. In theory user-defined representation classes could have other constraints, so a generalized mechanism would be good. This is certainly a solvable problem.

@mhvk
mhvk force-pushed the representations-more-powerful branch from 6bd2d73 to 7752d3c Compare September 8, 2016 17:59
@mhvk

mhvk commented Sep 8, 2016

Copy link
Copy Markdown
Contributor Author

OK, I think there is quite clear consensus that we should not have __eq__, so I removed that (and rebased as well, so this remains mergeable).

@adrn

adrn commented Sep 30, 2016

Copy link
Copy Markdown
Member

@mhvk Sorry for the delay, but I'd like to take a close look at this over the weekend.

@taldcroft taldcroft left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have finished another somewhat superficial review, scanning the code for things that jump out. I haven't look at the testing at all to see if is complete (@adrn?). But with those caveats I don't see any other problems with the code.

Comment thread CHANGES.rst
For JPL kernels, this roughly doubles the execution time, so if one requires
only the positions, one should use ``get_body_barycentric``. [#5231]

- Arithmetic on representations, such as addition of two representations,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest adding wording that this treats representation points as vectors in cartesian space. Then everything that follows is indeed "obvious".

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Done.

Comment thread CHANGES.rst

- Arithmetic on representations, such as addition of two representations,
multiplication with a ``Quantity``, or calculating the norm via ``abs``,
has now become possible. Furthermore, there are news methods ``mean``,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo news.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, done

Comment thread astropy/coordinates/representation.py Outdated
attr_classes = OrderedDict([('lon', Longitude),
('lat', Latitude)])
recommended_units = {'lon': u.deg, 'lat': u.deg}
_dimensional_representation = None # set to SphericalRepresentation below

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a good use for @classproperty in utils.decorators:

@classsproperty
def _dimensional_representation():
    return SphericalRepresentation

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, much nicer. Done.

Returns
-------
dot_product : `~astropy.units.Quantity`
The sum of the product of the x, y, and z components of ``self``

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy-paste mistake.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I seem to be blind to what actually is wrong here... Could you elucidate?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry, I was being blind and did not see the "of the product" words. So never mind.

Comment thread astropy/coordinates/representation.py Outdated
Returns
-------
dot_product : `~astropy.units.Quantity`
The sum of the product of the x, y, and z components of ``self``

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy-paste mistake

Comment thread astropy/coordinates/representation.py Outdated
and z components are calculated, and the result is converted back to
the same representation as the input.

Refer to `numpy.mean` for full documentation of the arguments.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This docstring and sum should still have a Returns section to document the return object.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, done

Comment thread astropy/coordinates/representation.py Outdated
that frame classes may override this with their own preferred units.
"""

attr_classes = OrderedDict()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this added? From looking at the code in MetaBaseRepresentation and docstring in BaseRepresentation, I think the intent is to require classes to explicitly define attr_classes (and not rely on a base class definition).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. Removing it doesn't seem to matter, so I did. I think it may be a leftover from an earlier attempt.

@mhvk
mhvk force-pushed the representations-more-powerful branch from e6fd632 to 76b695f Compare October 3, 2016 05:46
@mhvk

mhvk commented Oct 3, 2016

Copy link
Copy Markdown
Contributor Author

@taldcroft - I address your comments except that for one I wasn't quite sure what you meant with a "copy and paste" error (maybe I addressed it, but please check).

Also rebased, since this was getting a bit far behind. I think I'll call this ready for final review now...

@mhvk
mhvk force-pushed the representations-more-powerful branch from 3bd3fbb to f47e487 Compare October 21, 2016 21:03
@mhvk

mhvk commented Oct 21, 2016

Copy link
Copy Markdown
Contributor Author

Rebased now that #5254 is in.

@adrn

adrn commented Oct 23, 2016

Copy link
Copy Markdown
Member

I'm happy with this -- ok to merge 👍

@eteq eteq left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'm convinced this is a fine way to go, particularly given the power this approach demonstrates in #5434. I am a bit worried about this confusing users (i.e. that it's not element-wise but rather vector operations), so many of my inline comments are aimed at clearing that up... but I now agree that it's worth having this despite the confusion risk.

That said, I concur with @adrn's concern(and @taldcroft's maybe-objection?) to abs as it is implemented here. That is, to me it really seems more natural for abs to be element-wise absolute value, as that's what it means in a numpy context. That's not the same as what it is here, the 3-norm. And remember:"In the face of ambiguity, refuse the temptation to guess.". So I would advocate either of two approaches:

  1. Change abs to be element-wise cartesian abs (which then goes back to the self representation), and have the current abs be a norm method as @adrn advocated.
  2. Drop abs completely because its ambiguous (absbiguous?) and just use norm.

I slightly favor 2, but could go either way.

I also found two other problems while experimenting - I created mhvk#20 against @mhvk's branch with the regression tests for these:

  1. mul fails if the shape of the representation is not 1d. Some of the others do too, but I think they're all due to mul failing? Anyway, you can see in the tests @mhvk and maybe you can tell offhand where the problem is?
  2. abs does not give consistent types - sometimes its a Quantity and sometimes a Distance. But in either case right now, it actually yields a numpy array of object type, not a Quantity/Distance with the appropriate shape. I think the desired behavior is always a single Distance, so that's what the test checks for.

Comment thread astropy/coordinates/representation.py Outdated
for component in first.components))

def mean(self, *args, **kwargs):
"""Return the vector mean along a given axis.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest a slightly different wording here (and in sum, and in the other def mean/def sum's above and below) - "axis" could mean e.g. "x axis" or it could mean the axis keyword (which in the representation situation means subtly different things). So how about:

Returns the vector mean along each of the cartesian components.  Arguments like `axis` are for the axis of *each* component.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again a good point. I removed "along the given axis" as I agree it is only confusing, and expanded text as follows:

        """Vector mean.

        Averaging is done by converting the representation to cartesian, and
        taking the mean of the x, y, and z components. The result is converted
        back to the same representation as the input.

        Refer to `~numpy.mean` for full documentation of the arguments, noting
        that ``axis`` is the entry in the ``shape`` of the representation, and
        that the ``out`` argument cannot be used.

Comment thread astropy/coordinates/representation.py Outdated

def __abs__(self):
return np.sqrt(functools.reduce(
operator.add, (getattr(self, component)**2

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my narrative comment for my doubts about __abs__ in general... But one more specific item: why not just self.dot(self)**0.5? Or perhaps Distance(self.dot(self)**0.5, copy=False)? I think that will make it behave more consistently...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should avoid conversions to Distance - one might want to use CartesianRepresentations to represent velocities for example.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.dot is slower for representations other than Cartesian, since it first converts to Cartesian and then takes the product. Although this really only matters for Cylindrical since for the spherical ones I already override it. Still, my sense is to leave it as is.

Comment thread astropy/coordinates/representation.py Outdated

Returns
-------
mean : `~astropy.coordinates.BaseRepresentation`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here (and everywhere else below), maybe you should change `~astropy.coordinates.BaseRepresentation to same class as self or something like that? Right now it looks like it's saying the return type is always BaseRepresentation, which is of course wrong but isn't obvious to the poor user who doesn't fully understand the class heirarchy.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, will do.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to just representation throughout. This means one cannot click on the link in the docs, but I agree it is much clearer (and my guess is that people usually see the help within ipython anway by typing a ?)

Comment thread astropy/coordinates/representation.py Outdated
return self.to_cartesian().dot(other)

def cross(self, other):
"""Cross product of two representations.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps "vector cross product"? Cross product may not be ambiguous, but that makes it a bit clearer that this is the standard cross product irrespective of the current representation

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

Comment thread docs/coordinates/representations.rst Outdated
will be a subclass of `~astropy.coordinates.UnitSphericalRepresentation`.


Vector arithmetic

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I think this should go above "creating your own" - it's more likely that a user will want to know this than need to create their own...

Comment thread docs/coordinates/representations.rst Outdated
Vector arithmetic
^^^^^^^^^^^^^^^^^

Representations support basic vector arithmetic, in particular taking the

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's worth trying to be a bit more explicit here on what we mean by "basic vector arithmetic" - something like:

Representations support basic vector arithmetic - i.e., vector operations as applied to *cartesian* representations.
If the representation is not a `CartesianRepresentation`, the operation will be performed on the cartesian vector, but returned as the representation type of the first operand.

or similar. I realize that, strictly speaking, that's what "vector arithmetic" nominally means, but I think this subtly could be easily lost on a reader who isn't thinking carefully about this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to do this with the note at the very end, but now agree this would come too late. I thought a bit about changing the text, but ended up deciding to move the note higher up and make it a proper .. Note::, so it stands out in the documentation.

@mhvk

mhvk commented Oct 27, 2016

Copy link
Copy Markdown
Contributor Author

@eteq - thanks for the comments. I'm really confused about the failures, though: for me, both multiplication and abs work perfectly fine on multi-d data! Could you check that you really do get a problem with getting object dtype? (as pointed out by @StuartLittlefair, abs can and should sometimes return Quantity rather than Distance -- it only returns Distance for spherical coordinates with length units because for those the radius is initialized as a distance).

@mhvk

mhvk commented Oct 27, 2016

Copy link
Copy Markdown
Contributor Author

On __abs__ -- I still find it quite obvious, but as you are the third to disagree, I've now changed it to a norm method. I do see that it is an obvious choice as well, and an additional advantage is that it is easier to find. (I removed __abs__ altogether as I don't want anybody ever to take just absolute values of the components...)

@mhvk
mhvk force-pushed the representations-more-powerful branch from f47e487 to 8ee76c7 Compare October 27, 2016 16:00
@eteq

eteq commented Oct 27, 2016 •

Copy link
Copy Markdown
Member

for me, both multiplication and abs work perfectly fine on multi-d data!

Hmm, so if I add eteq/astropy@ec9cba8 (from mhvk#20) in on top of this branch as it stands right now, I get ValueError: operands could not be broadcast together with shapes (4,7) (3,1) failures when I try it locally. But if I take the tests as they stand in the branch right now, they all pass. I think this is because the problem is in __mul__, and right now it looks like __mul__ (and at least some of the other operations) don't get tested in more than 1D? That's why I added that test in eteq/astropy@ec9cba8, to make sure everything gets hit by the test suite in the n-D case.

Are you seeing it pass even if you try eteq/astropy@ec9cba8, @mhvk ?

Could you check that you really do get a problem with getting object dtype?

That problem, however, seems to have disappeared in the latest version, so all good there.

abs can and should sometimes return Quantity rather than Distance

Ah, good point! I get it now. I can see arguments one way or another for whether it should be forced either way, but I think it's probably best to leave it alone, as it's not clear which is better (and it's all Quantity-compatible anyway).

@mhvk

mhvk commented Oct 27, 2016

Copy link
Copy Markdown
Contributor Author

@eteq - yes, I tried your examples by hand, and they work on my machine. But just to be sure, in the latest commits I added a test (for __div__, but they use the same code path) on a 2-D array that the cross test had created anyway (see https://github.com/astropy/astropy/pull/5301/files#diff-3d38462c8d8bf432919ce1decd35579bR345) and that new tests also passes locally (sadly, don't know about travis!). Does the current PR pass for you as well? If it fails, I can only think it must be a python2 vs 3 issue; what python are you on?

@mhvk
mhvk force-pushed the representations-more-powerful branch from 8ee76c7 to 4df4d7e Compare October 28, 2016 13:25
@mhvk
mhvk merged commit 683aee3 into astropy:master Oct 28, 2016
@mhvk

mhvk commented Oct 28, 2016

Copy link
Copy Markdown
Contributor Author

@eteq - the new tests, including the 2-D array check, pass on travis for all settings. So, I went ahead and merged as I would like to rebase the usage to get that shipshape as well. If there still are problems with this PR, I can address them as part of #5434 (which I'll leave open until you're OK).

@mhvk
mhvk deleted the representations-more-powerful branch October 28, 2016 14:40
@eteq

eteq commented Oct 28, 2016

Copy link
Copy Markdown
Member

@mhvk - hmm, I'm still seeing this error in both py2 and py3. I'll make a PR into astropy just adding the test I described and lets see what travis tells us. That way we'll at least know if its local to my computer somehow, and can try to fix the error in that PR if so.

@taldcroft

Copy link
Copy Markdown
Member

🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants