 Check out the bookstore at IRBS.com |
|
|
|
|||||
|
||||||
Subject: GreatCircle Calculations - precision
From: Rino (jobbox@XXX.XXX)
Date: Mon Sep 30 2002 - 20:57:29 EDT
This is my first post on this list. I happened upon it when I did a search
for 'haversines' (an alien concept to me). I thought some of the posts were
very insightful and entertaining, and I hope I can get an answer to some of
my questions:
Last weekend I decided to convert an old navigation spreadsheet application
I had built in my seafaring days to a VB.NET solution.
It basically became a full rewrite; first because the spreadsheet used old
Excel macro's and not VBA, and second because I hadn't given a thought to
spherical triangles, Mr. Neper's rules and various other laws, in over ten
years.
In rewriting the whole thing in .NET, I used 'double' (64-bit floating
point) precision data type for all variables (Latitude, Longitude, course,
etc). I did this because I use the value PI, which is an .NET constant of
type 'Double', and I wanted to preserve that precision in my calculations.
When testing my code, I found that my results varied slightly from the
answers given in one of my old textbooks, from results returned by my old
casio programmable calculator, and also from my spreadsheet. I am 99%
certain that my code is accurate, I checked all intermediate results, and
they all checked out.
So, my question to those of you with a better grasp of math (and programming
perhaps), is: why do I see these differences? Is the textbook correct, or
my code? The differences are negligible in outcomes for course and
distance, but they have a more significant impact on the Vertex (probably
because the slight difference in course propagated through the formulas for
the vertex).
I will include an example here:
Lat1: 34 40.0N Lon1:139 55.0E
Lat2: 37 45.0N Lon2:122 50.0W
GreatCircle distance:
TextBook: 4473.8
VB.NET: 4473.8
GreatCircle course:
TextBook: 54.5
VB.NET 54.3
Vertex:
TextBook: 47 59.4N 168 36.8W
VB.NEt: 48 06.0N 168 26.2W
I would appreciate any insights from the 'brains' on this list.
Thanks.
Rino
|