Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

Re: zsh converts a floating-point number to string with too much precision



On Fri, Dec 20, 2019 at 5:59 PM Stephane Chazelas
<stephane.chazelas@xxxxxxxxx> wrote:
>
> 2019-12-20 02:37:11 +0100, Vincent Lefevre:
> > With zsh 5.7.1, I get:
> >
> > zira% echo $((1.1))
> > 1.1000000000000001
> >
> > because zsh seems to first select the precision independently
> > from the value, i.e. 17 to be able to convert the string back
> > to floating point, preserving the original value, then it
> > outputs the closest number in this precision.
> >
> > Instead, zsh should select the minimum precision so that the
> > inverse conversion can give the original value, i.e. it should
> > output 1.1 here.
>
> And what should it give for
>
> $((1.1000000000000001)) ?
>
> (hint, 1.1000000000000001 and 1.1 have the same "double"
> representation).

I think what Vincent meant is that zsh should produce the shortest
string that, when parsed, results in a value equal to the original.

For your example, "1.1" is the shortest string that parses into
floating point value equal to the original, hence this (according to
Vincent) is what zsh should produce.

Many languages and libraries do this sort of thing. The roundtrip
guarantee is sometimes limited to the same machine. That is, some
implementation don't guarantee that you can serialize a floating point
value on one machine, parse it on another and get the same value.

Roman.



Messages sorted by: Reverse Date, Date, Thread, Author