Zsh Mailing List Archive
Messages sorted by:
Reverse Date,
Date,
Thread,
Author
Re: Floating point calculus error...
- X-seq: zsh-users 5290
- From: Vincent Lefevre <vincent@xxxxxxxxxx>
- To: David Kågedal <davidk@xxxxxxxxxxxxxx>
- Subject: Re: Floating point calculus error...
- Date: Sun, 25 Aug 2002 20:07:08 +0200
- Cc: Patrick Aussems <zyk@xxxxxxxxxxxxxx>, zsh-users@xxxxxxxxxxxxxx
- In-reply-to: <87k7me2921.fsf@xxxxxxxxxx>
- Mailing-list: contact zsh-users-help@xxxxxxxxxx; run by ezmlm
- References: <1030288941.517.8.camel@Amok> <20020825155349.GD6783@xxxxxxxxxxxxxx> <87k7me2921.fsf@xxxxxxxxxx>
On Sun, Aug 25, 2002 at 18:32:38 +0200, David Kågedal wrote:
> Vincent Lefevre <vincent@xxxxxxxxxx> writes:
> > $ echo $[0.4]
> > 0.40000000000000002
> >
> > I think that it should give the minimum number of decimals in such a way
> > that when converting back to binary, one should get the same result.
>
> But it does.
No, this is not the *minimum* number of decimal digits. 0.4 would
be sufficient in this case.
--
Vincent Lefèvre <vincent@xxxxxxxxxx> - Web: <http://www.vinc17.org/> - 100%
validated (X)HTML - Acorn Risc PC, Yellow Pig 17, Championnat International
des Jeux Mathématiques et Logiques, TETRHEX, etc.
Work: CR INRIA - computer arithmetic / SPACES project at LORIA
Messages sorted by:
Reverse Date,
Date,
Thread,
Author