Repository navigation
BUG: not assigning full precision values to longdouble scalars #4381
Description
If you try to set a longdouble scalar with a string, it appears to go through Python's normal string-parsing and therefore gets converted to a C double rather than a long double:
In [19]: numpy.longdouble("1.12345678901234567890")
Out[19]: 1.1234567890123456912
Note the loss of those last couple digits.
In [20]: numpy.float("1.12345678901234567890")
Out[20]: 1.1234567890123457
In [21]: numpy.longdouble("1.12345678901234567890") - numpy.float("1.12345678901234567890")
Out[21]: 0.0
And so internally they are identical.
One solution to this is a special case for longdoubles and to use strtold() to do the conversion.
Cheers,
Scott
Activity
charris commentedon Feb 27, 2014
Yeah, this is a long standing problem. There may even be other issues opened for it.
juliantaylor commentedon Feb 27, 2014
this one is in principle easy to fixi n LONGDOUBLE_setitem, if we could assume strtold is available
unfortunately this is only guaranteed by C99
aarchiba commentedon Aug 12, 2015
sscanf could be used, in principle anyway: K&R describes %Lf as a valid scanf speicifier, so that should work in every reasonable C environment.
charris commentedon Aug 12, 2015
BSD has a
strtold. We could just steal it.njsmith commentedon Aug 12, 2015
Worth testing on windows, sure, in case of tickling some library weirdness
with scanf. But since on windows long double is the same as double then it
can't go that wrong.
On Aug 12, 2015 11:23 AM, "Julian Taylor" [email protected] wrote:
aarchiba commentedon Aug 13, 2015
A BSD-licensed implementation would be nice to include on all platforms, actually, because strtol is locale-dependent and I don't think we want that. One could perhaps modify python's float/string conversion routines, though it looks messy. Or there's netlib's gdtoa (at http://www.netlib.org/fp/ ). But for the moment I'd rather have precision-retaining but locale-dependent functions than the reverse.
pv commentedon Aug 13, 2015
Locale-dependen functions are nasty --- consider
1,2345vs1.2345and on european locales there's either total loss of precision or errors. Although we do evil things already.aarchiba commentedon Aug 13, 2015
Python internally uses the dtoa function from netlib; I think then that it is probably reasonable for numpy to include the gdtoa functions from netlib (same author and algorithm but platform- and locale-independent support for long double, quad, double double, and other possibilities). This handles conversions number->string as well as string->number.
charris commentedon Aug 13, 2015
Sounds like we might need a documented policy on this somewhere.
aarchiba commentedon Aug 16, 2015
I'm not sure we need a policy - python's str() and repr() produce numbers in a documented, locale-independent way. So do % and format(), IIRC; if you want anything locale-y you have to explicitly use the locale-aware functions. Naturally numpy should follow the same rule. A reminder might be good, though.
aarchiba commentedon Aug 26, 2015
I have fixed all the string->long double conversions I could think of in PR #6199 . The long double -> string conversions are trickier, since some of them go through python's format method, so it would be necessary to convert format specifiers into % specifiers and then use snprintf_l or the like. But repr works and doesn't lose precision.
BUG: fix numpy#4381: precision loss on string -> longdouble conversion