I'm looking at the freely available Solar potential dataset on Google BigQuery that may be found here: https://bigquery.cloud.google.com/table/bigquery-public-data:sunroof_solar.solar_potential_by_censustract?pli=1&tab=schema
Each record on the table has the following border definitions:
lat_max - maximum latitude for that region
lat_min - minimum latitude for that region
lng_max - maximum longitude for that region
lng_min - minimum longitude for that region
Now I have a coordinate (lat/lng pair) and I would like to query to see whether or not that coordinate is within the above range. How do I do that with BQ Standard SQL?
I've seen the Geo Functions here: https://cloud.google.com/bigquery/docs/reference/standard-sql/geography_functions
But I'm still not sure how to write this query.
Thanks!
Assuming the points are just latitude and longitude as numbers, why can't you just do a standard numerical comparison?
Note: The first link doesn't work without a google account, so I can't see the data.
But if you want to become spatial, I'd suggest you're going to need to take the border coordinates that you have and turn them into a polygon using one of: ST_MAKEPOLYGON, ST_GEOGFROMGEOJSON, or ST_GEOGFROMTEXT. Then create a point using the coords you wish to test ST_MAKEPOINT.
Now you have two geographies you can compare them both using ST_INTERSECTION or ST_DISJOINT depending on what outcome you want.
If you want to get fancy and see how far aware from the border you are (which I guess means more efficient?) you can use ST_DISTANCE.
Agree with Jonathan, just checking if each of the lat/lon value is within the bounds is simplest way to achieve it (unless there are any issues around antimeridian, but most likely you can just ignore them).
If you do want to use Geography objects for that, you can construct Geography objects for these rectangles, using
ST_MakePolygon(ST_MakeLine(
[ST_GeogPoint(lon_min, lat_min), ST_GeogPoint(lon_max, lat_min),
ST_GeogPoint(lon_max, lat_max), ST_GeogPoint(lon_min, lat_max),
ST_GeogPoint(lon_min, lat_min)]))
And then check if the point is within particular rectangle using
ST_Intersects(ST_GeogPoint(lon, lat), <polygon-above>)
But it will likely be slower and would not provide any benefit for this particular case.
Related
I have an oracle database (11g spatial) that includes a series of area polygons and water mains. I'm trying to attribute each of these mains to the area in which it is contained and for the most part this is straightforward enough (using the SDO_CONTAINS function) but I'm not sure how to deal with mains that straddle multiple polygons due to errors in digitisation.
In cases like this what I'd ideally like to do is attribute a main to an area polygon if the majority of it's length (>50%) is contained within onit. I know that I can use the SDO_RELATE function to determine every polygon that any given main interacts with, but I don't know how to then go about determining how much of it's length is contained within each area.
The principle is like this:
Correlate mains and areas. Assuming you have many mains and many areas, the most efficient approach is to use SDO_JOIN
For each couple (main/area) returned, compute their intersection (SDO_GEM.SDO_INTERSECTION) and measure the length of that intersection (SDO_GEOM.SDO_LENGTH).
From those results, retain the area for each main where the length is the maximum
If you want a full SQL example, allow me a bit of time to write that using sample data.
I have an application with a sqlite database that contains 7000+ records in it with city names, longitudes and latitudes.. also these "cities" are connected to relevant city fields on the database too.
What my app doing is, query the current location with core location, fetch the lon and lat values, and then find the closest location from the database.
The result doesn't have to be super accurate (i just want to match cities), so I want to use Hypotenuse formula for finding the closest point:
closest city in db: min((x1-x2)^2 +(y1-y2)^2)^(1/2)
x1, y1: lon and lat for user
x2, y2: lon and lat for points in database.
If I was using ms-sql or sqlite database, I could easily create a query but when it comes to core data, I'm out of ideas.
I don't want to fetch all the data (and fill the memory) then aggregate this formula on all fields so is there a way to create a query and get the result from the db?
Am I overthinking this problem, and missing a simple solution?
If I'm understanding your problem correctly, you're wanting to find the closest "n" cities to your current location.
I had something similar and here's how I approached it.
In essence, you probably need to take each city's lat/lon and hash it into some index. We use a Mercator Projection to convert the lat/lon to x/y, then hash that value in a manner similar to how Google/Bing/Apple Maps hash their map tiles. Fortunately, MapKit has a built-in Mercator Projection function.
In pseudocode:
for each city's lat/lon {
CLLocationCoordinate2D coordinate = (CLLocationCoordinate2D){lat, lon};
MKMapPoint point = MKMapPointForCoordinate(coordinate);
//256 represents the size of a map tile at zoomLevel 20. You can use whatever zoomLevel
//you want here, but we need something to quickly lookup close-by cities.
//this is the formula you can use to determine how granular your index is
//(256 * pow(2, (20 - zoomLevel)))
NSInteger x = point.x/256.0;
NSInteger y = point.y/256.0;
save x & y in a CityHashIndex table
}
Now, you get the current location's lat/lon, hash that into the index as above, and just simply write a query against this CityHashIndex table.
So say that, for simplicity sake, you're current location is indexed at 1000, 1000. So to find close by cities, maybe you search for cities with indexes in the range of `900-1100, 900-1100'.
From there, you're now only pulling in a much smaller set of cities and the memory requirements to process your Hypotenuse Formula isn't so bad.
I can elaborate more if you're interested.
This is directly related to a commonly asked question about Core Data.
Searching for surrounding suburbs based on latitude & longitude using Objective C
Calculate a bounding box around the point you need (min lat/long max lat/long) then use an NSPredicate against those values to find everything within the box. From there you can do a distance calculation on the results that return and sort them.
I would suggest setting this up so that it can search at multiple distances then you can see if a city is within 10 miles, 100 miles, etc. Slowly increasing the bounding box until you get one or more results back.
I would use NSPredicate to define my search criteria it will act as a filter. I'm not sure how optimized is this and if it will pull all your registers but I'm assuming that coreData has some kind of indexing mechanism that will optimize the search.
You can take a look of this document
https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/CoreData/Articles/cdFetching.html
Check the section named
Retrieving Specific Objects
There seems to be no standard whether Longitude,Latitude or Latitude,Longitude should be used.
WSG84 and stuff based directly on it, seem to prefer Long,Lat.
"Normal people" always tend to speak of Lat, Long - so I've very often seen code or frameworks that use Lat, Long (e.g. google Maps)
Is there any strong argument for either way?
You are correct, there is no universal standard on the order:
In mathematical functions which do an universal conversion, between x,y or lon,lat or inverse, the lon,lat order should be used, because the x-axis relates to longitude and y to latitude and the x,y order is usually preferred.
Further, if you program a piece of code which is related to draw a lon,lat coordinate on x,y coordinates (screen), I also would use the lon,lat order because of the direct relation to x,y.
The order lat,lon is the classical one, coming from (old) navigation and geography. I assume that latitude in that field is used first because it was easier to measure (using only a ruler, the sun and a stick for length of shadow measuring).
The longitude was not determinable for long time. If you read old adventure reports, they only tell the latitude that their expeditions reached).
I think therefore they use latitude first, the measurement of longitude came later in history, once precise chronometers have been available and transportable.
So for apps that display coordinates info on a screen you should display latitude first.
As mentioned in the comments by #Midavalo, there is a standard for the representation of geographic locations by coordinates: ISO 6709.
It describes that a geographical point is specified by the following four items:
a first horizontal coordinate (y), such as latitude
a second horizontal coordinate (x), such as longitude
optionally, a vertical coordinate, i.e. height or depth
optionally, an identification of the coordinate reference system (CRS)
The order, positive direction, and units of coordinates are supposed to be defined by that CRS, but when such a CRS identification is missing -- which is very often -- the data must be interpreted by the following conventions:
Latitude comes before longitude
North latitude is positive
East longitude is positive
Fraction of degrees is preferred over sexagesimal (degrees, minutes, seconds) notation
So, unless another coordinate reference system is mentioned, the standard is "latitude, longitude, (elevation)".
There seems to be no standard whether Longitude,Latitude or Latitude,Longitude should be used.
The problem is the exact opposite: there are several competing standards. There's ISO 6709 which specifies Lat-Long and is followed by the EPSG:4326 Geodetic Parameter to represent coordinates on the World Geodetic System. But there's also the CRS:84 parameter which uses the same coordinate system but with inverted axis (Long-Lat).
It's a matter of choosing which standard to follow rather than lacking standardization. Still, that doesn't mean that all choices are equally convenient.
Is there any strong argument for either way?
Yes, prefer Lat-Long for GIS data unless you're catering to a specific audience or use case. As mentioned, most people default to Lat-Long and many GIS applications will too, so if you have no reason to prefer Long-Lat, stick to Lat-Long. Otherwise, it might be better to just pick whatever convention your tools will favor. For instance, Long-Lat is the choice for GeoJSON so if you're primarily processing GeoJSON data, it's your call to decide if it's worth it to convert back and forth. If you're not using GIS software and just want to store coordinates as if they were (x,y) points on a plane, Long-Lat is more intuitive - but beware the risks of rolling your own geodesic calculations instead of using a proper GIS library to interpret coordinate reference systems.
I have a coordinate, assume any latitude and longitude values.
I have a domain setup on SimpleDB that has many items (simple strings) with attributes of 'Latitude' and 'Longitude'. Now what I want to do is query SimpleDB and see if the current location coordinates are 'x' meters apart from SimpleDB's items' coordinates. 'x' should be 10.
My app uploads an item to SimpleDB with an attribute that contains the latitude and longitude. I detect the users location, get the coordinates and I want to use a select expression to see if the coordinates are 'x' meters apart. So is their a better approach to doing this? Or is this is the best way, if so, how can I do it?
Here is an example of what the select expression may look like, I just have no idea how to use it in this case and what the '%#' values would be filled in by. This whole format could be off, its just my idea.
select * from test-app-simpledb where Latitude >= '%#' AND Latitude <= '%#' AND Longitude >= '%#' AND Longitude <= '%#'
So "test-app-simpledb" is my SimpleDB domain name, Latitude and Longitude are the attributes I compare to the coordinates. They are all converted to string.
So, how can I do location comparative select expressions. Querying if the item's coordinate (latitude and longitude attribute) are 'x' (in this case 10) meters apart.
Any way to do this? Thanks!
In the revised question, it has become clear that the actual question is how to conduct a query against a Amazon SimpleDB database to see if locations in the database are within a certain radius of a location provided by an iOS app.
As you correctly identify, you really want to do this server-side, rather than client-side, if possible. And the particular solutions will be highly dependent upon the particular database technology (SimpleDB in this case).
This question is touched upon in Spatial queries on AWS SimpleDB. I would suggest checking that out for more information.
As a proxy for a proper distance algorithm, you could translate the distance in meters into a ranges of latitudes and longitudes. Thus, you could, in iOS, calculate a minimum and maximum for both latitude and longitude and then pass those along in the WHERE clause to your remote database. Then the server could filter results based upon those criteria. That admittedly gives you a square-shaped region (rather than a circular region that you get by calculating distances properly), but it makes it really easy to quickly limit the result set with no special geolocation logic required on the server. To do this, you could define a region with MKCoordinateRegionMakeWithDistance, and then grab its span.
MKCoordinateRegion region = MKCoordinateRegionMakeWithDistance(originalCoordinate, 200, 200);
MKCoordinateSpan span = region.span;
That gives you span.latitudeDelta and span.longitudeDelta which you can center around a given location's latitude and longitude to define a square shaped region around a location. To demonstrate that in action, here the center pin is my location at Times Square, and the shaded area is a region (constructed using the above span which is 200m wide and tall), which covers longitudes between 100m east and 100m west of my current location, as well as latitudes 100m north and 100m south of my current location.
This is a way to employ MapKit.framework functions to greatly streamline your SQL queries for remote databases to filter locations based upon geographic distance. If the square region is problematic, you could also further filter the results once they're downloaded to the iOS app using the CLLocation instance method, distanceFromLocation to determine the actual distance. But by limiting the longitudes and latitudes of locations retrieved by the server, you dramatically streamline the server retrieval process while not encumbering it with too much geographic location calculations.
But you really want to have SimpleDB do the full, proper distance calculation, I'll have to leave that to others.
Below, is my original answer. In the original question, I misinterpreted it as being "how do I construct a list of coordinates going in a circle around a particular location?" As made clear by the revised question, that was not the issue at all, but I'll keep my old answer here for historical reference.
Original answer:
If you used Calculate new coordinate x meters and y degree away from one coordinate, the implementation might look like:
NSInteger numberOfPoints = 10;
for (double bearing = 0.0; bearing < 360.0; bearing += (360.0 / numberOfPoints))
{
CLLocationCoordinate2D coordinate = [self coordinateFromCoord:originalCoordinate
atDistanceKm:distanceKm
atBearingDegrees:bearing];
// do whatever you want with this coordinate
}
It seems to work fine. For example, I had an app use this routine to drop 10 pins 100m from me in Times Square:
We have an application that has a database full of polygons (currently stored as points) that a .net app pulls out and checks if they overlap.
I occurred to me that it would be much nicer to convert these point arrays to polygon / polyline objects within the database and use sql to get a bool of weather they overlap or not.
I have seen different methods suggested to do this but non of the examples given were quite in-line with my needs.
I would be very happy to receive input from those kind enough to offer their experience.
Additional:
In response to questions: It is indeed 2D. and yes any crossover of the two is considered true. The polygons have n points and can be concave. The polygons will be saved as 1 per row (after data conversion task) as polygons (i.e. the polygon type .. it might be called something else spatial / geom my memory is not on my side right now)
You can use .STIntersection with .STAsText() to test for overlapping polygons. (I really hate the terminology Microsoft has used (or whoever set the standard terms). "Touching," in my mind, should be a test for whether or not two geometry/geography shapes overlap at all, not just share a border.)
Anyway....
If #RadiusGeom is a geometry representing a radius from a point, the following will return a list of any two polygons where an intersection (a geometry that represents the area where two geometries overlap) is not empty.
SELECT CT.ID AS CTID, CT.[Geom] AS CensusTractGeom
FROM CensusTracts CT
WHERE CT.[Geom].STIntersection(#RadiusGeom).STAsText() <> 'GEOMETRYCOLLECTION EMPTY'
If your geometry field is spatially indexed, this runs pretty quickly. I ran this on 66,000 US CT records in about 3 seconds. There may be a better way, but since no one else had an answer, this was my attempt at an answer for you. Hope it helps!
Calculate and store the bounding rectangle of each polygon in a set of new fields within the row which is associated with that polygon. (I assume you have one; if not, create one.) When your dotnet app has a polygon and is looking for overlapping polygons, it can fetch from the database only those polygons whose bounding rectangles overlap, using a relatively simple SQL SELECT statement. Those polygons should be relatively few, so this will be efficient. Then, your dotnet app can perform the finer polygon overlap calculations in order to determine which ones of those really overlap.
Okay, I got another idea, so I am posting it as a different answer. I think my previous answer with the bounding polygons probably has some merit on its own, even if it was to reduce the number of polygons fetched from the database by a small percentage, but this one is probably better.
MSSQL supports integration with the CLR since version 2005. This means that you can define your own data type in an assembly, register the assembly with MSSQL, and from that moment on MSSQL will be accepting your user-defined data type as a valid type for a column, and it will be invoking your assembly to perform operations with your user-defined data type.
An example article for this technique on the CodeProject: Creating User-Defined Data Types in SQL Server 2005
I have never used this mechanism, so I do not know details about it, but I presume that you should be able to either define a new operation on your data type, or perhaps overload some existing operation like "less-than", so that you can check if one polygon intersects another. This is likely to speed things up a lot.