December 11, 2024, 09:30:57 PM

News:

IonicWind Snippit Manager 2.xx Released!  Install it on a memory stick and take it with you!  With or without IWBasic!


Questionnable results with Setprecision

Started by pistol350, October 04, 2007, 07:01:24 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

pistol350

Hi all!
i'm just messing around with math operations.while dealing with miles to kilometers conversion and
after setting setprecision to 10 digits , i received a "weird" result.
I mean, see the code below, the result i expected was : 1.6093440000

Any idea on the reasons why i get 1.6093440056 instead ?
/* miles to kilometers */
/* Thursday Oct 4th 2007 */
/* pistol350 */
REM compile as console window
' 1 mile = 1,609344 kilometers
'CONST fConv = 1.609344
/******************************************/
SETPRECISION 10
$OPTION "FLOAT"
REM These will all be defined as type FLOAT
fConv = 1.609344
fOutput = 0
fResults = 0
/******************************************/
OPENCONSOLE
input "enter miles value to receive the output in kilometers: ",fOutput
fResults = (fOutput*fConv)
print "results in kilometers : ",fResults
print "press a key to quit"
do:until INKEY$<>""
closeconsole
END
Regards,

Peter B.

ExMember001

October 04, 2007, 07:24:50 AM #1 Last Edit: October 04, 2007, 07:27:25 AM by KrYpT
here's how i do it in one of my program ;)


/* miles to kilometers */
/* Thursday Oct 4th 2007 */
/* krypt :D */
REM compile as console window
/******************************************/
float fOutput
float fResults
/******************************************/
OPENCONSOLE
input "enter miles value to receive the output in kilometers: ",fOutput
fResults = (fOutput / 0.6214);
print "results in kilometers : ",USING("%f#.# KM",fResults)
print "press a key to quit"
do:until INKEY$<>""
closeconsole
END


but i dont know how to do that your way  ???

ExMember001


GWS

Tut, tut Mr Pistol .. :)  haven't you read any of my notes on variable precision ..

Using 'Float' variables you can only rely on 7 significant figures.  If you need more, you have to use type 'Double'  ..  :)

all the best,

Graham
Tomorrow may be too late ..

pistol350

October 04, 2007, 12:47:02 PM #4 Last Edit: October 04, 2007, 12:58:29 PM by pistol350
Lol!  ;D
Thank you Krypt for the working example and the tips.
I Thank you as well Graham for hitting me with a rod  ::)
LOL
we sometimes need to be taught with the old school ways to be a hardworking student  ;)

PS: There is a ";" in line 11 Into Krypt's code that should not be there.
Aurora,Aurora,Aurora..... ;D
Just a note  ;)
Regards,

Peter B.

GWS

Tomorrow may be too late ..

ExMember001

yep your right  :D
but does nothing bad ... didnt even notice ..hehe