Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

Re: zsh converts a floating-point number to string with too much precision



On 12/20/19, Vincent Lefevre <vincent@xxxxxxxxxx> wrote:
> With zsh 5.7.1, I get:
>
> zira% echo $((1.1))
> 1.1000000000000001
>
> because zsh seems to first select the precision independently
> from the value, i.e. 17 to be able to convert the string back
> to floating point, preserving the original value, then it
> outputs the closest number in this precision.
>
> Instead, zsh should select the minimum precision so that the
> inverse conversion can give the original value, i.e. it should
> output 1.1 here.

You can use typeset -F1 one=1.1 to specify the output precision of a
parameter (note that this doesn't affect the float value stored, you
can change to -F20 later to display more decimals without
reassignment). So in your case you could count the number of digits in
the string after the . and then pass that to -F if you wanted to.

-- 
Mikael Magnusson



Messages sorted by: Reverse Date, Date, Thread, Author