I have a ROUTES table which has columns SOURCE_AIRPORT and DESTINATION_AIRPORT and describes a particular route that an airplane would take to get from one to the other.
I have an AIRPORTS table which has columns LATITUDE and LONGITUDE which describes an airports geographic position.
I can join the two tables using columns which they both share called SOURCE_AIRPORT_ID and DESTINATION_AIRPORT_ID in the routes table, and called IATA in the airports table (a 3 letter code to represent an airport such as LHR for London Heathrow).
My question is, how can I write an SQL query using all of this information to find, for example, the longest route out of a particular airport such as LHR?
I believe I have to join the two tables, and for every row in the routes table where the source airport is LHR, look at the destination airport's latitude and longitude, calculate how far away that is from LHR, save that as a field called "distance", and then order the data by the highest distance first. But in terms of SQL syntax i'm at a loss.
This would have probably been a better question for the Mathematics Stack Exchange, but I’ll provide some insight here. If you are relatively farmiliar with trigonometry, I’m sure you could understand the implementation given this resource: https://en.m.wikipedia.org/wiki/Haversine_formula. You are looking to compute the distance between two point on the surface of a sphere in terms of their distance across its surface (not a straight line, you can’t travel through the Earth).
The page displays this formula:
https://wikimedia.org/api/rest_v1/media/math/render/svg/a65dbbde43ff45bacd2505fcf32b44fc7dcd8cc0
Where
• φ1, φ2 are the latitude of point 1 and latitude of point 2 (in radians),
• λ1, λ2 are the longitude of point 1 and longitude of point 2 (in radians).

If you data is in degrees, you can simply convert to radians by multiplying by pi/180
There is a formula called great circle distance to calculate distance between two points. You probably can load is as a library for your operating system. Forget the haversine, our planet is not a perfect sphere.
If you use this value often, save it in your routes table.
I think you're about 90% there in terms of the solution method. I'll add in some additional detail regarding a potential SQL query to get your answer. So there's 2 steps you need to do to calculate the distances - step 1 is to create a table containing joining the ROUTES table to the AIRPORTS table to get the latitude/longitude for both the SOURCE_AIRPORT and DESTINATION_AIRPORT on the route. This might look something like this:
SELECT t1.*, CONVERT(FLOAT, t2.LATITUDE) AS SOURCE_LAT, CONVERT(FLOAT, t2.LONGITUDE) AS SOURCE_LONG, CONVERT(FLOAT, t3.LATITUDE) AS DEST_LAT, CONVERT(FLOAT, t3.LONGITUDE) AS DEST_LONG, 0.00 AS DISTANCE_CALC
INTO ROUTE_CALCULATIONS
FROM ROUTES t1 LEFT OUTER JOIN AIRPORTS t2 ON t1.SOURCE_AIRPORT_ID = t2.IATA
LEFT OUTER JOIN AIRPORTS t3 ON t1.DESTINATION_AIRPORT_ID = t3.IATA;
The resulting output should create a new table titled ROUTE_CALCULATIONS made up of all the ROUTES columns, the longitude/latitude for both the SOURCE and DESTINATION airports, and a placeholder DISTANCE_CALC column with a value of 0.
Step 2 is calculating the distance. This should be a relatively straightforward calculation and update.
UPDATE ROUTE_CALCULATIONS
SET DISTANCE_CALC = 2 * 3961 * asin(sqrt((sin(radians((DEST_LAT- SOURCE_LAT) / 2))) ^ 2 + cos(radians(SOURCE_LAT)) * cos(radians(DEST_LAT)) * (sin(radians((DEST_LONG- SOURCE_LONG) / 2))) ^ 2))
And that should give the calculated distance in the DISTANCE_CALC table for all routes seen in the data. From there you should be able to do whatever distance-related route analysis you want.
Related
I have a Postgres table with some data created by using a shapefile. I need to find all records which intersect within 2 miles radius from a given latitude and longitude. I used some queries including the following one.
SELECT * FROM us_census_zcta WHERE ST_INTERSECTS(geom,
CIRCLE(POINT(40.730610, -73.935242), 2));
But none of them worked. What I am doing wrong here? How can I get the results I need?
The SRID is 4269 (NAD 83).
EDIT: After Mike Organek pointed me out that I have switched the lat-long in the above query. And then I tried a few things and the following query gave me 1 record.
SELECT * FROM us_census_zcta WHERE ST_INTERSECTS(geom::geometry,
ST_SETSRID(ST_POINT(-73.935242, 40.730610), 4269)::geometry);
But how can I use Circle and find records which intersect within 2 miles radius from that given lat-long?
What you're looking for is ST_DWithin, which will check if records intersect within a given buffer. In order to use it with miles you better cast the geometry column to geography, as it then computes distances in metres:
For geography: units are in meters and distance measurement defaults to use_spheroid=true. For faster evaluation use use_spheroid=false to measure on the sphere.
SELECT * FROM us_census_zcta
WHERE
ST_DWithin(
geom::geography,
ST_SetSRID(ST_MakePoint(-73.935242, 40.730610), 4269)::geography,
3218.688); -- ~2 miles
Keep in mind that this cast might affect query performance if the indexes aren't set properly.
See also: Getting all Buildings in range of 5 miles from specified coordinates
I have three SQL tables:
A list of 100k weather stations with a latitude and longitude coordinate
A list of 15 cities of interest with a latitude and longitude coordinate
A list of weather data for each weather station
My interest right now is only with the first two tables. How do I filter the list of weather stations to those within e.g. 100km of each city of interest?
I have a Microsoft SQL Server and I'd prefer to do it within SQL if possible.
Basically, if you try to do this yourself, you end up with a Cartesian product:
select c.*, ws.*
from cities c cross apply
(select ws.*
from (select ws.*,
<complicated expression to calculate distance> as distance
from weather_station ws
) ws
where distance < 100
) ws;
In order to get the list of weather stations, all cities and weather stations have to be compared. The distance calculation is often rather expensive, so you can cut down on this by "prefiltering". For instance, in most inhabited places, 100 km is within 1 degree latitude and 2 degrees longitude:
select c.*, ws.*
from cities c cross apply
(select ws.*
from (select ws.*,
<complicated expression to calculate distance> as distance
from weather_station ws
where ws.latitutde between c.latitude - 1 and c.latitude + 1 and
ws.longitude between c.longitude - 2 and c.longitude + 2
) ws
where distance < 100
) ws;
Although that helps, this is still essentially a filtered Cartesian product.
So, what should you really do? If you care about coordinates as spatial data, you should look into SQL Server's spatial extensions (the documentation is here, particularly the geography type because that is most relevant to your needs).
As Gordon mentioned, you can define spatial geography datatype for your needs. You can follow below steps to achieve the goal.
Store the latitude, longitude data in the Point
Now, use the STDistance to calculate the distance between two points
You can leverage common scenario of finding nearest neighbor
I have a bunch of stores with their own Latitudes and Longitudes. I'm trying to pull data that is within a 2 mile radius of each point. Eg. How many stores are within 2 miles of each store. What is the best way to go about this?
I know rounding the lat/longs to the tenth (18.4, -66.2) can essentially give me 5 mile radius, but how do I get more granular. I'm not sure how granular rounding to the 100th (18.4, -66.21) gets me in terms of miles, but seems too small of a radius.
Date is stored as:
Store Name (string)
Latitude (double)
Longitude (double)
What you want is spatial join:
https://prestodb.io/blog/2020/05/07/local-spatial-joins
Just join a table with itself, on condition that distance between two points is below 2 miles, and aggregate. Something like this:
SELECT
a.store_name,
(COUNT(*) - 1) AS neighbors -- subtract 1 for self
FROM stores a JOIN stores b
ON ST_Distance(ST_Point(a.longitude, a.latitude),
ST_Point(b.longitude, b.latitude)) < 2 * 1609
GROUP BY a.store_name
Make sure you have a relatively fresh Presto installation, I think Presto got it optimized around end of 2018, and it would run as plain cross join before that - which would be too slow.
I'm trying to have a go at learning about PostgreSQL and in particular, it's PostGIS extension and the benefits with regards to geographic spatial features it provides. I've loaded a PostgreSQL DB with a table that contains 30,000 records of latitude, longitude and a price value (for houses) and I want to start querying the DB to return all the rows that would be in a radius of Xkm of a particular latitude and longitude.
I've hit a brick wall as to how I might run this type of query as I've found the documentation to be quite limited online and I've found no similar attempts at this method of querying online.
Some methods I've tried:
SELECT *
FROM house_prices
WHERE ST_DWithin( ST_MakePoint(53.3348279,-6.269547099999954)) <= radius_mi *
1609.34;
This prompts the following error:
ERROR: function st_dwithin(geometry) does not exist
Another attempt:
SELECT * FROM house_prices ST_DWithin( 53.3348279, -6.269547099999954, 5); <-- A latitude value, longitude value and 5 miles radius
This prompts the following error:
ERROR: syntax error at or near "53.3348279"
Could anyone point me in the right direction/ know of some documentation I could look at?
** Edit **
Structure and set up of database and table in pgAdmin4
The first query has an invalid number of parameters. The function ST_DWithin expects at least two geometries and the srid distance,
and optionally a Boolean parameter indicating the usage of a spheroid (see documentation).
The second query is missing a WHERE clause and has the same problem as the first query.
Example from documentation:
SELECT s.gid, s.school_name
FROM schools s
LEFT JOIN hospitals h ON ST_DWithin(s.the_geom, h.the_geom, 3000)
WHERE h.gid IS NULL;
Perhaps something like this would be what you want to achieve:
SELECT *
FROM house_prices h
WHERE ST_DWithin(ST_MakePoint(53.3348,-6.2695),h.geom,h.radius_mi * 1609.34)
Also pay attention to the order of the coordinates pair (x,y or y,x), otherwise you might easily land on the sea with these coordinates ;-)
EDIT： Taking into account that there is no geometry on the table, so the points are stored in two different columns, longitude and latitude:
SELECT *
FROM house_prices
WHERE ST_DWithin(ST_MakePoint(longitude,latitude),ST_MakePoint(53.3348,-6.2695),1609.34)
I am trying to get the count of all records within 50 miles of each record in a huge table (1m + records), using self join as shown below:
proc sql;
create table lab as
select distinct a.id, sum(case when b.value="New York" then 1 else 0 end)
from latlon a, latlon b
where a.id <> b.id
and geodist(a.lat,a.lon,b.lat,b.lon,"M") <= 50
and a.state = b.state;
This ran for 6 hours and was still running when i last checked.
Is there a way to do this more efficiently?
UPDATE: My intention is to get the number of new yorkers in a 50 mile radius from every record identified in table latlon which has name, location and latitude/longitude where lat/lon could be anywhere in the world but location will be a person's hometown. I have to do this for close to a dozen towns. Looks like this is the best it could get. I may have to write a C code for this one i guess.
The geodist() function you're using has no chance of exploiting any index. So, you have an algorithn that's O(n**2) at best. That's gonna be slow.
You can take advantage of a simple fact of spherical geometry, though, to get access to an indexable query. A degree of latitude (north - south) is equivalent to sixty nautical miles, 69 statute miles, or 111.111 km. The British definition of nautical mile was originally equal to a minute. The original Napoleonic meter was defined as one part in ten thousand of the distance from the equator to the pole, also defined as 90 degrees.
(These defintions depend on the assumption that the earth is spherical. It isn't, quite. If you're a civil engineer these definitions break down. If you use them to design a parking lot, it will have some nasty puddles in it when it rains, and will encrooach on the neighbors' property.)
So, what you want is to use a bounding range. Assuming your latitude values a.lat and b.lat are in degrees, two of them are certainly more than fifty statute miles apart unless
a.lat BETWEEN b.lat - 50.0/69.0 AND b.lat + 50.0/69.0
Let's refactor your query. (I don't understand the case stuff about New York so I'm ignoring it. You can add it back.) This will give the IDs of all pairs of places lying within 50 miles of each other. (I'm using the 21st century JOIN syntax here).
select distinct a.id, b.id
from latlon a
JOIN latlon b ON a.id<>b.id
AND a.lat BETWEEN b.lat - 50.0/69.0 AND b.lat + 50.0/69.0
AND a.state = b.state
AND geodist(a.lat,a.lon,b.lat,b.lon,"M") <= 50
Try creating an index on the table on the lat column. That should help performance a LOT.
Then try creating a compound index on (state, lat, id, lon, value). Try those columns in the compound index in different orders, if you don't get satisfactory performance acceleration. It's called a covering index, because the some of its columns (the first two in this case) are used for quick lookups and the rest are used to provide values that would otherwise have to be fetched from the main table.
Your question is phrased ambiguously - I'm interpreting it as "give me all (A, B) city pairs within 50 miles of each other." The NYC special case seems to be for a one-off test - the problem is not to (trivially, in O(n) time) find all cities within 50 miles of NYC.
Rather than computing Great Circle distances, find Manhattan distances instead, using simple addition, and simple bounding boxes. Given (A, B) city tuples with Manhattan distance less than 50 miles, it is straightforward to prune out the few (on diagonals) that have Great Circle (or Euclidean) distance less than 50 miles.
You didn't show us EXPLAIN output describing the backend optimizer's plan.
You didn't tell us about indexes on the latlon table.
I'm not familiar with the SAS RDBMS. Oracle, MySQL, and others have geospatial extensions to support multi-dimensional indexing. Essentially, they merge high-order coordinate bits, down to low-order coordinate bits, to construct a quadtree index. The technique could prove beneficial to your query.
Your DISTINCT keyword will make a big difference for the query plan. Often it will force a tablescan and a filesort. Consider deleting it.
The equijoin on state seems wrong, but maybe you don't care about the tri-state metropolitan area and similar densely populated regions near state borders.
You definitely want the WHERE clause to prune out b rows that are more than 50 miles from the current a row:
too far north, OR
too far south, OR
too far west, OR
too far east
Each of those conditionals boils down to a simple range query that the RDBMS backend can evaluate and optimize against an index. Unfortunately, if it chooses the latitude index, any longitude index that's on disk will be ignored, and vice versa. Which motivates using your vendor's geospatial support.