Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

Floating-point arithmetic evaluation on Linux/x86 (traditional FPU)

The zshmisc man page says:

  The  shell  can  perform  integer and floating point arithmetic, either
  using the builtin let, or via a substitution of the form $((...)).  For
  integers,  the  shell is usually compiled to use 8-byte precision where
  this is available, otherwise precision is 4 bytes.  This can be tested,
  for example, by giving the command `print - $(( 12345678901 ))'; if the
  number appears unchanged, the precision is at least 8 bytes.   Floating
  point arithmetic is always double precision.

But it is not true that zsh always uses double precision. It seems to
use the C double type, but this doesn't correspond to double precision
by default under Linux/x86 (when the traditional FPU is used, i.e. no
SSE2). For instance, I get:

vin:~> uname -a
Linux vin 2.6.22-1-686-bigmem #1 SMP Sun Jul 29 15:54:01 UTC 2007 i686 GNU/Linux
vin:~> echo $((9007199254740994.0+(1.0-1/65536.0)-9007199254740994.0))

instead of "0." on other machines, due to the "double rounding"
effect. More information about this problem on:


Either zsh should switch to double precision (assuming that modules
and so on won't change the precision back to the extended precision),
or the zsh documentation should be changed.

Vincent Lefèvre <vincent@xxxxxxxxxx> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Messages sorted by: Reverse Date, Date, Thread, Author