You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Nice find! The problem is in objint_mpz.c, in the definition of mp_maxsize_obj. I think the .len field is wrong (line 61). Try making .len=2 on 64-bit unix.
Making .len=2 on 64-bit unix does indeed give correct behavior. What preprocessor expression should I use in place of MPZ_NUM_DIG_FOR_INT to give the correct value across platforms?
I think the problem with the MPZ_NUM_DIG_FOR_INT macro was the +1 in it's definition. Removing it makes the number of digits stay at 2 for 32/64 bit unix. Let me know if it's okay to open a pull req.
@Alex-March I just beat you to it! In this case the calculation was trying to round the division up, so the correct fix is (+ MPZ_DIG_SIZE-1) before the division. Also I fixed the definition of sys.maxsize to make it more robust.
Activity
dpgeorge commentedon Aug 31, 2015
Nice find! The problem is in objint_mpz.c, in the definition of mp_maxsize_obj. I think the .len field is wrong (line 61). Try making .len=2 on 64-bit unix.
pramasoul commentedon Aug 31, 2015
Making .len=2 on 64-bit unix does indeed give correct behavior. What preprocessor expression should I use in place of
MPZ_NUM_DIG_FOR_INTto give the correct value across platforms?dpgeorge commentedon Sep 1, 2015
I don't know, it has to be worked out. Maybe the definition of MPZ_NUM_DIG_FOR_INT can be fixed.
hosaka commentedon Sep 15, 2015
I think the problem with the MPZ_NUM_DIG_FOR_INT macro was the +1 in it's definition. Removing it makes the number of digits stay at 2 for 32/64 bit unix. Let me know if it's okay to open a pull req.
py/mpz: Fix calculation of max digit storage for mpz; fix sys.maxsize.
dpgeorge commentedon Sep 15, 2015
Fixed in 8b4fb4f.
@Alex-March I just beat you to it! In this case the calculation was trying to round the division up, so the correct fix is (+ MPZ_DIG_SIZE-1) before the division. Also I fixed the definition of sys.maxsize to make it more robust.
Yes, you are always welcome to open a PR :)
hosaka commentedon Sep 15, 2015
@dpgeorge good timing, thanks!
py/mpz: Fix calculation of max digit storage for mpz; fix sys.maxsize.
Merge pull request micropython#1448 from dhalbert/sphinx-fixes