Floating point problem in SQL Server 2005 -


The code below is used to calculate the miles between the two cities in this case, it Yarmouth, Yarmouth from ME is for distance from ME - clearly zero - meaning that the results within the X miles of Yarmouth should be included in Yarmouth only.

The problem is that the latitude and longitude is to create some sort of floating point problems Ermvt (I have not seen it with other cities):

  DECLARE @ Fromlang float, @ Flomlet float @ Lolon float, @ Potot float, @ Test float SET @fromlong = 43.8219 SET @fromlat = -70.1758 SET @ Tioelong = 43.8219 SET @ Tioelti = -70kl758 SET @ Teaist = Saian (@ Fromlang / (180 / Pi ()) SIN (@TOL / / (180 / pi ())) + COS (@fromlang / (180 / pi ())) * cos (@ Speak / (180 / pi ())) * cos (@ Flomlet / (180 / pi ()) - @ Tolat / (180 / PI ()) Print / *** "1" *** / SELECT 3963.0 * ACOS (@test) / *** show "displays a domain error" *** /  

the first, it is a SQL server bug?

second, I can do to get around it? I know the above example I if @test & gt; There may be some argument for; 1 , but this example is not available in the web app (not like me) Is distilled from a query went to Bed, so I query must fix, ie precisely calculated, if the resort Tiskyuel possible, and any thoughts do not return values ​​distorted to another?

Comments are used to agree with float, I used float only in the example because these columns The datatype was where the latitude / longitude was being read, but it seems to be a problem. Since I can not change the datatypes of columns themselves, the simplest solution was to change the datatype of the resulting calculation, which works fine in all cases:

  SELECT 3963.0 * ACOS (convert ( DCIMAL (10, 6), TTES))  

I have missed out on calculations that there may be small round errors in the form of float, but they are acceptable for this application Are there. Everyone comments.


Comments

Popular posts from this blog

Eclipse CDT variable colors in editor -

AJAX doesn't send POST query -

wpf - Custom Message Box Advice -