Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

zsh converts a floating-point number to string with too much precision



With zsh 5.7.1, I get:

zira% echo $((1.1))
1.1000000000000001

because zsh seems to first select the precision independently
from the value, i.e. 17 to be able to convert the string back
to floating point, preserving the original value, then it
outputs the closest number in this precision.

Instead, zsh should select the minimum precision so that the
inverse conversion can give the original value, i.e. it should
output 1.1 here.

FYI, GNU MPFR has the same issue with mpfr_printf and %Re (with
an empty precision field), and I regard this as a bug:
  https://sympa.inria.fr/sympa/arc/mpfr/2019-12/msg00000.html
  https://sympa.inria.fr/sympa/arc/mpfr/2019-12/msg00001.html

Note that Java does it right:

zira:~> cat tst.java
public class tst
{
  public static void main(String[] args)
  {
    double x;
    x = 0x1.1999999999999p+0;
    System.out.println(x);
    x = 0x1.199999999999ap+0;
    System.out.println(x);
    x = 0x1.199999999999bp+0;
    System.out.println(x);
  }
}
zira:~> javac tst.java
zira:~> java tst
1.0999999999999999
1.1
1.1000000000000003

-- 
Vincent Lefèvre <vincent@xxxxxxxxxx> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)



Messages sorted by: Reverse Date, Date, Thread, Author