Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

Re: arithmetic anomaly

• X-seq: zsh-users 26609
• From: Ray Andrews <rayandrews@xxxxxxxxxxx>
• To: zsh-users@xxxxxxx
• Subject: Re: arithmetic anomaly
• Date: Fri, 26 Mar 2021 07:36:21 -0700
• Archived-at: <https://zsh.org/users/26609>
• List-id: <zsh-users.zsh.org>

```On 2021-03-26 3:35 a.m., Roman Perepelitsa wrote:
```
```
```
Maybe this will help: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html <https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html>
```

Good to know!

```
Seriously, I've often wondered what really goes on in a calculator. Ok, for now I know that zsh and probably everybody else will compute endless reams of garbage if you let it.  Here's my algorithm with the outer 'for' loop increasing exponentially:
```
Ray's  Algorithm fractional:
Level: 1: win: 1 / 1
Level: 10: win: 6513215599 / 10000000000
```
Level: 100: win: 6339676587267708385630188881009247729557115329795338127068495312693456362833186743509112446
```7602830066897486524892436064552875964380756727929631216242621046676674030227216977589219271
```
483398068889452544 / 9999999999999999697331222125103616594745032754550236264824175095034684843555407553419633840
```4706251868027512415973882408182135734368278484639385041047239877871023591066789981811181813
306167128854888448
Level: 1000: win: inf / inf
Level: 10000: win: inf / inf
Level: 100000: win: inf / inf
Level: 1000000: win: inf / inf

```
.... I'm asking why the 'Level: 100' result isn't just aborted after 15 digits given that all the rest is garbage?
```

```
```
```
Understanding how floating point numbers work is very useful. It's virtually mandatory when writing numerical computation code. None of this is specific to zsh.
```
```
Amen!  Seriously, I had no idea this sort of thing was permitted. Thanks Roman.  For now I'll use ' %d' for output and at least I'll know when things turn to mud, there must be limits to precision and there must be overflow at some point.  (But of course we hafta use floats if there's a division anywhere, just don't use them for display.)
```

```

Messages sorted by: Reverse Date, Date, Thread, Author