Pete's QBASIC Site

I have a string that equals 9889.98 when I use the VAL function to convert it to a number I get 9889.979999999 Why is that? I'm converting lots of numbers this way and I always get one or two that does this.. It's throwing my totals off.

I can't duplicate the error. Try printing the string and VAL return.

Are you using variables like: value = VAL(number$) ?

Undefined numerical variables default as Single, but it seems like a Double problem.

I am pulling the number from a comma delimited file by using the MID$ command, AMT$ = MID$(data$,2,7) then all I do is add it to a totals variable like this: Total# = Total# + VAL(AMT$). Funny thing is, it only happens once in a while, there's no rhyme or reason. When I print out the string AMT$ it equals 9889.89, once it convert it using VAL(AMT$) and print I get 9889.8899999. Could it be a variable issue?

Most likely due to the fact that there's no possible 32 bit pattern for 9889.89, so it settles with the closest that it can find: 9889.8899999. Some decimal numbers (like 0.1 IIRC) are repeating decimals in binary, so it would take an infinite amount of bits to be right. Since that's not possible (we use 32 bits or 64 bits for floating point numbers), the computer approximates to the nearest possible number it can find. My guess is that this is one of those numbers.

You are using Double variables which creates the longer value accuracy.

num$ = "9889.98"

value = VAL(num$) ' Single value is accurate

TOTAL# = TOTAL# + value ' returns TOTAL as a Double value

PRINT value, TOTAL#

Certain values, not all, will do that. If you are adding a lot of values, you can convert the results back to a string to get rid of the extra decimal places.

Or you can display the trimmed results with:

PRINT USING " #############,.##"; TOTAL#

PRINT USING will also round the decimal places, but that is just for show. The actual value does not change.

burger,
You are correct. I changed my code and it corrected my VAL issue. But that brought about another problem, when I do the
TOTAL# = TOTAL# + value ---- it returns TOTAL as a Double value, but brings me right back to my original problem that I had. 9889.89 turns into 9889.8896484375 for some reason...

My total will be in the millions with a decimal, do I need to define it as a Double?

Crap.. this can't be this hard!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
All I want to add is add large numbers together....

You can use Double values. When the total is done, use a string. Single values may not work if they are over 7 digit numbers:

sum$ = LTRIM$(STR$(TOTAL#))

'Use INSTR to find the decimal point place:

position% = INSTR(sum$, ".") + 2 ' for 2 decimal places

result$ = MID$(sum$, 1, position%)

PRINT result$