An invalid floating point operation occurred when checking longitude and latitudes - sql

Question

I know there are a lot of questions like mine asked. I looked at them, but I can't seem to resolve the problem. I'm just bad in math :s.
In c# i'm using a SqlQuery to get cities in a range. This works most of the time, except for some cities.
For example, if i ask the nearby cities of the city with postalcode 2060. I get this error. When i ask it for the city with code 2000, it returns 2000 and 2060.
2060: Lat = 51.2293515000 Long = 4.4279883000
2000: Lat = 51.2198771000 Long = 4.4011356000
this is the query:
return base.Database.Database.SqlQuery<City>(
"SELECT * FROM [dbo].[City] WHERE #p0 >= (((acos(sin((#p1*pi()/180)) *
sin(([Latitude]*pi()/180))+cos((#p1*pi()/180)) * cos(([Latitude]*pi()/180)) *
cos(((#p2- [Longitude])*pi()/180))))*180/pi())*60*1.1515*1.609344)"
, radius, latitude, longitude);
Can somebody explain how this can be changed so it works for all the "cities" and the reason if this error for a noob in math?
Thank you

Answer 1

The problem is that the statement inside acos() gets a rounding error somtimes when you compare cities to themselves, and the statement becomes slightly over 1, which causes the error.
You need to cap the value to max 1 somehow. Here is a rather ugly solution:
return base.Database.Database.SqlQuery<City>(
"SELECT * FROM [dbo].[City] WHERE #p0 >= (((acos((select min(v) from (values (1), (sin((#p1*pi()/180)) *
sin(([Latitude]*pi()/180))+cos((#p1*pi()/180)) * cos(([Latitude]*pi()/180)) *
cos(((#p2- [Longitude])*pi()/180)))) as x(v))))*180/pi())*60*1.1515*1.609344)"
, radius, latitude, longitude);
I'm not sure, but it could be that it sometimes gets slightly smaller than -1 as well, which would case the same error. Then you'd have to cap the value to between -1 and 1.