Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Galculator math errors?

Here you can discuss every aspect of Debian. Note: not for support requests!
Post Reply
Message
Author
No_windows
Posts: 505
Joined: 2015-08-05 03:03

Galculator math errors?

#1 Post by No_windows »

updated 64bit Jessie, using Galculator.

I type .30 x 100 =
I get 30
I type -30 =
I get -1.11022302463e-15

What?????

srq2625
Posts: 44
Joined: 2016-02-26 11:01

Re: Galculator math errors?

#2 Post by srq2625 »

No_windows wrote:updated 64bit Jessie, using Galculator.

I type .30 x 100 =
I get 30
The decimal point causes the calculation to be performed in floating-point math. The result is displayed as a fixed-point approximation of the floating-point result (which is correct).
No_windows wrote: I type -30 =
I get -1.11022302463e-15
What I'm guessing happened here is the calculation is still being done in floating-point, hence the "error". What might have been a better (I'm guessing here, no testing of the idea has been done) coding is to take the display value (which has only an implied decimal point) and proceed using fixed point math.

tynman
Posts: 131
Joined: 2016-05-03 19:48
Location: British Columbia, Canada
Been thanked: 1 time

Re: Galculator math errors?

#3 Post by tynman »

This caught my attention because I use galculator fairly regularly. Maybe I need to find a more accurate calculator. Something I actually need to worry about, as opposed to all the recent nonsense FUD in the news and forums about CPU "flaws". Oh but wait, "E-15" .... :)

User avatar
RU55EL
Posts: 546
Joined: 2014-04-07 03:42
Location: /home/russel

Re: Galculator math errors?

#4 Post by RU55EL »

Galculator works perfectly in RPN mode. At least, that is the mode I always use. (It is also easier for me to use in RPN mode.)

.3 [enter]
100 [*]
Display shows: 30
30 [-]
Display shows: 0

Post Reply