VAL function problem
VAL function problem
I have a string that equals 9889.98 when I use the VAL function to convert it to a number I get 9889.979999999 Why is that? I'm converting lots of numbers this way and I always get one or two that does this.. It's throwing my totals off.
- burger2227
- Veteran
- Posts: 2466
- Joined: Mon Aug 21, 2006 12:40 am
- Location: Pittsburgh, PA
I can't duplicate the error. Try printing the string and VAL return.
Are you using variables like: value = VAL(number$) ?
Undefined numerical variables default as Single, but it seems like a Double problem.
Are you using variables like: value = VAL(number$) ?
Undefined numerical variables default as Single, but it seems like a Double problem.
Please acknowledge and thank members who answer your questions!
QB64 is a FREE QBasic compiler for WIN, MAC(OSX) and LINUX : https://www.qb64.org/forum/index.php
Get my Q-Basics demonstrator: https://www.dropbox.com/s/fdmgp91d6h8ps ... s.zip?dl=0
QB64 is a FREE QBasic compiler for WIN, MAC(OSX) and LINUX : https://www.qb64.org/forum/index.php
Get my Q-Basics demonstrator: https://www.dropbox.com/s/fdmgp91d6h8ps ... s.zip?dl=0
I am pulling the number from a comma delimited file by using the MID$ command, AMT$ = MID$(data$,2,7) then all I do is add it to a totals variable like this: Total# = Total# + VAL(AMT$). Funny thing is, it only happens once in a while, there's no rhyme or reason. When I print out the string AMT$ it equals 9889.89, once it convert it using VAL(AMT$) and print I get 9889.8899999. Could it be a variable issue?
Most likely due to the fact that there's no possible 32 bit pattern for 9889.89, so it settles with the closest that it can find: 9889.8899999. Some decimal numbers (like 0.1 IIRC) are repeating decimals in binary, so it would take an infinite amount of bits to be right. Since that's not possible (we use 32 bits or 64 bits for floating point numbers), the computer approximates to the nearest possible number it can find. My guess is that this is one of those numbers.
For any grievances posted above, I blame whoever is in charge . . .
- burger2227
- Veteran
- Posts: 2466
- Joined: Mon Aug 21, 2006 12:40 am
- Location: Pittsburgh, PA
I see what happened...
You are using Double variables which creates the longer value accuracy.
num$ = "9889.98"
value = VAL(num$) ' Single value is accurate
TOTAL# = TOTAL# + value ' returns TOTAL as a Double value
PRINT value, TOTAL#
Certain values, not all, will do that. If you are adding a lot of values, you can convert the results back to a string to get rid of the extra decimal places.
Or you can display the trimmed results with:
PRINT USING " #############,.##"; TOTAL#
PRINT USING will also round the decimal places, but that is just for show. The actual value does not change.
num$ = "9889.98"
value = VAL(num$) ' Single value is accurate
TOTAL# = TOTAL# + value ' returns TOTAL as a Double value
PRINT value, TOTAL#
Certain values, not all, will do that. If you are adding a lot of values, you can convert the results back to a string to get rid of the extra decimal places.
Or you can display the trimmed results with:
PRINT USING " #############,.##"; TOTAL#
PRINT USING will also round the decimal places, but that is just for show. The actual value does not change.
Please acknowledge and thank members who answer your questions!
QB64 is a FREE QBasic compiler for WIN, MAC(OSX) and LINUX : https://www.qb64.org/forum/index.php
Get my Q-Basics demonstrator: https://www.dropbox.com/s/fdmgp91d6h8ps ... s.zip?dl=0
QB64 is a FREE QBasic compiler for WIN, MAC(OSX) and LINUX : https://www.qb64.org/forum/index.php
Get my Q-Basics demonstrator: https://www.dropbox.com/s/fdmgp91d6h8ps ... s.zip?dl=0
burger,
You are correct. I changed my code and it corrected my VAL issue. But that brought about another problem, when I do the
TOTAL# = TOTAL# + value ---- it returns TOTAL as a Double value, but brings me right back to my original problem that I had. 9889.89 turns into 9889.8896484375 for some reason...
My total will be in the millions with a decimal, do I need to define it as a Double?
Crap.. this can't be this hard!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
All I want to add is add large numbers together....
You are correct. I changed my code and it corrected my VAL issue. But that brought about another problem, when I do the
TOTAL# = TOTAL# + value ---- it returns TOTAL as a Double value, but brings me right back to my original problem that I had. 9889.89 turns into 9889.8896484375 for some reason...
My total will be in the millions with a decimal, do I need to define it as a Double?
Crap.. this can't be this hard!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
All I want to add is add large numbers together....
- burger2227
- Veteran
- Posts: 2466
- Joined: Mon Aug 21, 2006 12:40 am
- Location: Pittsburgh, PA
You can use Double values. When the total is done, use a string. Single values may not work if they are over 7 digit numbers:
sum$ = LTRIM$(STR$(TOTAL#))
'Use INSTR to find the decimal point place:
position% = INSTR(sum$, ".") + 2 ' for 2 decimal places
result$ = MID$(sum$, 1, position%)
PRINT result$
sum$ = LTRIM$(STR$(TOTAL#))
'Use INSTR to find the decimal point place:
position% = INSTR(sum$, ".") + 2 ' for 2 decimal places
result$ = MID$(sum$, 1, position%)
PRINT result$
Please acknowledge and thank members who answer your questions!
QB64 is a FREE QBasic compiler for WIN, MAC(OSX) and LINUX : https://www.qb64.org/forum/index.php
Get my Q-Basics demonstrator: https://www.dropbox.com/s/fdmgp91d6h8ps ... s.zip?dl=0
QB64 is a FREE QBasic compiler for WIN, MAC(OSX) and LINUX : https://www.qb64.org/forum/index.php
Get my Q-Basics demonstrator: https://www.dropbox.com/s/fdmgp91d6h8ps ... s.zip?dl=0