Two On A Big Ocean The Story of the First Circumnavigation
of the Pacific Basin
in a Small Sailing Ship


      

Other Books by
Hal Roth
| Home | Mailing Lists | Bookstore | Weather | Tide Predictions | Bowditch |

refraction formula comparison


Subject: refraction formula comparison
From: Paul Hirose (71202.2014@XXX.XXX)
Date: Sun Oct 01 2000 - 16:36:50 EDT


The Explanatory Supplement to the Astronomical Almanac gives this
low-precision formula for refraction below 15 degrees:

R = (P / (T + 273)) * (.1594 + .0196 * H + .00002 * H^2) / (1 + .505
* H + .0845 * H^2)

where H = apparent altitude, P = air pressure in millibars, and T =
degrees Celsius. Result is in degrees.

The Explanatory Supplement also gives first-order approximations to
correct celestial coordinates for refraction. I will not give them
here, but they are simple formulas. You could use them to determine
the refraction-corrected separation angle between stars.

That was some sharp thinking by George Huxtable, to spot the problem
lurking in the Bennett refraction formula. I'd expect the exact
location of the singularity to depend on your machine. With my HP-15C
calculator, it's at 89.92249993. Any change in the last decimal place
avoids it.

George is quite correct that a programmer should not shrug off a
potential bomb like this. I once crashed an astronomy program by
innocently entering exactly 90 degrees for a parameter.

Here is a comparison of the refraction formulas:

 H Bennett Supp. Keys Smart

 0 34.5 34.1 35.0 -
 1 24.3 24.1 25.4 151.1
 2 18.2 18.1 18.8 2.0
 5 9.9 9.8 9.9 9.5
10 5.4 5.3 5.3 5.3
12 4.5 4.4 4.5 4.5
15 3.6 3.6 3.6 3.6
30 1.7 1.8 1.7 1.7

Except for formula no. 2, all are equal at higher altitudes.

The Explanatory Supplement formula is only supposed to be used below
15 degrees, and the Smart formula from 15 up, but both are good for
navigation outside those ranges. I included 12 degrees because that's
where the maximum error of the Bennett formula is supposed to occur.





| Home | Mailing Lists | Bookstore | Weather | Tide Predictions | Bowditch | Trawlerworld |