While testing code to convert geographical positions in string format to decimal format, I thought everything was ok... until I did the same calulations usingmy calclulator for a quality check. I always get rather large differences in the decimals whencomparing calulated values with calculator values.
This was supposed to be simple, but I must be doing something wrong and am unable to see what. Any explanations?
Hopefully, the comments in the code explain what I'm trying to do
' PosConv.cba - convert positions in string format to decimal format
'
' TEST POS: 54 27 17 N - Calculator answer: 54.45472222
' TEST POS: 22 53 06 S - Calculator answer: -22.88499999
'
DECLARE PosConv(degrees:string,minutes:string,seconds:string,dval:string)
SETPRECISION = 8
OPENCONSOLE
PRINT PosConv("54","27","17","N")
PRINT PosConv("22","53","06","S")
DO:UNTIL inkey$<>""
CLOSECONSOLE
END
SUB PosConv(degrees:string,minutes:string,seconds:string,dval:string)
'----------------------------------------------------------------------------------------
' Converts from degrees, minutes and seconds to decimal degrees
' Decimal degrees = degrees + (minutes/60) + (seconds/3600)
' Make separate calls for latitude and longitude.
' South latitudes and/or West longitudes are negative values
'----------------------------------------------------------------------------------------
def ret,tmp:float
ret = val(degrees)
tmp = val(minutes)
ret = ret + tmp/60
tmp = val(seconds)
ret = ret + tmp/3600
if dval = "S" then ret = -ret
if dval = "W" then ret = -ret
RETURN ret
Regards,
Egil.
My first guess, use DOUBLE instead of FLOAT.
Bill
Egil,
If Creative works the same way as IWB then you need to add in the return type in your declare line:
DECLARE PosConv(degrees:string,minutes:string,seconds:string,dval:string),double
And the same for the sub...
SUB PosConv(degrees:string,minutes:string,seconds:string,dval:string),double
def ret,tmp:double
Bill is correct about using Double instead of Float, I tried both and Double was by far the best.
It does however return a slightly different result to your calculator (at least in IWB):
54.45472222 > same as your calc
-22.88500000 > -22.88499999 - very close.
Whilst testing my mathematical functions in StringMap, I found many differences in the answers from on-line calculators, when I looked deeper into this I found there are different algorithms used in some languages, hence different results.
IWB version:
' PosConv.cba - convert positions in string format to decimal format
'
' TEST POS: 54 27 17 N - Calculator answer: 54.45472222
' TEST POS: 22 53 06 S - Calculator answer: -22.88499999
'
DECLARE PosConv(degrees:string,minutes:string,seconds:string,dval:string),DOUBLE
SETPRECISION(8)
OPENCONSOLE
PRint PosConv("54","27","17","N")
PRint PosConv("22","53","06","S")
DO:UNTIL inkey$<>""
CLOSECONSOLE
END
SUB PosConv(degrees:string,minutes:string,seconds:string,dval:string),DOUBLE
'----------------------------------------------------------------------------------------
' Converts from degrees, minutes and seconds to decimal degrees
' Decimal degrees = degrees + (minutes/60) + (seconds/3600)
' Make separate calls for latitude and longitude.
' South latitudes and/or West longitudes are negative values
'----------------------------------------------------------------------------------------
def ret,tmp:DOUBLE
ret = val(degrees)
tmp = val(minutes)
ret = ret + tmp/60
tmp = val(seconds)
ret = ret + tmp/3600
if dval = "S" then ret = -ret
if dval = "W" then ret = -ret
RETURN ret
endsub
Using Float gave me big differences, with Double at least you could find your way home!
Andy.
:)
Thanks guys!
DOUBLE instead of FLOAT did the trick. And the difference on example number two is just becauce the CB version rounds the answer and my calculator don't.
Almost feeling ashamed here, this is the second time I do the same mistake (float instead of double). Maybe I'm too old for this??? ???
Andy:
CB is more "forgiving" than the other IW languages. Subroutine declarations can be with or without declared expected return types. The results are alwlays the same. But if you do declare the return type, it has to be done this way:
DECLARE PosConv(degrees:string,minutes:string,seconds:string,dval:string) as double
Otherwise you get a messagebox saying "Variable wrong type in line xx".
Regards,
Egil.
"Almost feeling ashamed here, this is the second time I do the same mistake"
Egil, if I had a Pound, Dollar or Euro for everytime I have done the same I would be a millionaire.
Andy.
QuoteAlmost feeling ashamed here, this is the second time I do the same mistake
Must admit that the last word was missing in that statement, that was "today".
Managed to trap it the first time yesterday, but not when doing the same mistake two hours later. Kept me busy for hours before I posted my "mayday".... :-[