Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

Re: arithmetic anomaly

On 2021-03-26 3:35 a.m., Roman Perepelitsa wrote:

Maybe this will help: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html <https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html>

Good to know!

Seriously, I've often wondered what really goes on in a calculator. Ok, for now I know that zsh and probably everybody else will compute endless reams of garbage if you let it.  Here's my algorithm with the outer 'for' loop increasing exponentially:

Ray's  Algorithm fractional:
Level: 1: win: 1 / 1
Level: 10: win: 6513215599 / 10000000000
Level: 100: win: 6339676587267708385630188881009247729557115329795338127068495312693456362833186743509112446
483398068889452544 / 9999999999999999697331222125103616594745032754550236264824175095034684843555407553419633840
Level: 1000: win: inf / inf
Level: 10000: win: inf / inf
Level: 100000: win: inf / inf
Level: 1000000: win: inf / inf

.... I'm asking why the 'Level: 100' result isn't just aborted after 15 digits given that all the rest is garbage?

Understanding how floating point numbers work is very useful. It's virtually mandatory when writing numerical computation code. None of this is specific to zsh.

Amen!  Seriously, I had no idea this sort of thing was permitted. Thanks Roman.  For now I'll use ' %d' for output and at least I'll know when things turn to mud, there must be limits to precision and there must be overflow at some point.  (But of course we hafta use floats if there's a division anywhere, just don't use them for display.)

Messages sorted by: Reverse Date, Date, Thread, Author