So, I've got some data that has longitude and latitude. I don't know what projection those are from. I've got some latitude and longitude I'll be fetching from Google maps API, which uses a projection with SRID of 3857.
If I just assume the data is from the same projection, and it turns out they're not, how far off could my distances be?
For instance, if they're from a 3-d projection (say 4326), but I just put them into a Geometry column with SRID 3857, and we're in the Northern Hemisphere, (Great Lakes area, but also other parts of the US), is there a way I can figure out how far off that would be?
EPSG:3857 uses meters as units, while EPSG:4326 uses degrees. If you try to plot them on the same map without reprojecting one or the other, they will be very far off (many orders of magnitude) from each other.
You said you'll be fetching lat-lng from the Google Maps API, using a EPSG:3857 as a projection, but latitude and longitude coordinates are not projected by definition, although they may use a different datum. I can't find official Google documentation, but consensus seems to be that Google Maps API uses WGS84, same as EPSG:4326, so lat-lngs you pull from google maps API will probably fit exactly on top of others from EPSG:4326.
See http://spatialreference.org/ref/epsg/4326/ and http://spatialreference.org/ref/sr-org/7483/ and https://gis.stackexchange.com/questions/34276/whats-the-difference-between-epsg4326-and-epsg900913
Related
I would like to calculate driving distance between two points writing SQL in Google BigQuery. I understand there is a method to calculate linear distance or "bird" miles using the following function: ROUND(ST_DISTANCE(ST_GEOGPOINT(C.LONGITUDE, C.LATITUDE), ST_GEOGPOINT(B.LNG_NBR, B.LAT_NBR))/1609.34,2) AS LINEAR_DIST_MILES
However, I am interested in driving distance instead of a linear distance. Is there a way to do this natively in Google BigQuery without needing to hit a Google Map API? I've also explored some solutions in R but that requires a Google Maps API key.
You would need two parts
good roads datasets
routing algorithms
BigQuery public datasets includes OpenStreetMaps, which is a reasonable dataset of roads (and other types of information) in most areas. There is also TIGER (bigquery-public-data.geo_us_roads) dataset which is US-specific.
Carto provides a sets of UDFs that can be used for routing. They've published an article how to connect things together:
https://carto.com/blog/how-to-do-route-optimization-at-scale-with-carto-bigquery/
I am working on a project where I save the Latitude and Longuite of a vehicle each an interval. I have also a route saved as an array of gps coordinates. So I would like to know if there is some library, that helps me to know if a point is inside the rout and other basic calculations with the coordinates as distance calculations for ex.
Any tool an any language helps!
Based on your comment, since you're not building a typical internet map, I might recommend you use a combination of Python and the Shapely library. You can see some nice examples on this post over at GIS.SE.
GIS Analyses: Geometry Types, Buffering, Intersection, etc.
In order to treat several individual Lat/Long positions as a "route", you'll need to format them as points in a LineString geometry type. Also beware: In most GIS software, points are arranged as X,Y. That means you'll be adding your points as Long,Lat. Inverting this is a common mistake that can be frustrating if you're not aware of it.
Next, in order to test whether any given point is within your route, you'll need to Buffer your route (LineString). I would use the accuracy of the GPS unit, + a few extra meters, as my buffering radius. This will give you a proper geometry (Polygon) for a Point-In-Polygon test (i.e. Intersection) that will calculate whether a given point is within the bounds of the route.
The GIS.SE post I linked to provides examples for both buffering and intersection using Python and Shapely.
Some notes about coordinates: Geodetic vs. Cartesian
I'm not confident if Shapely will perform reliable calculations on geodetic data, which is what we call the familiar coordinates you get from GPS. Before doing operations in Shapely, you may need to translate your long/lat points into projected X/Y coordinates for an appropriate coordinate system, such as UTM, etc. (Hopefully someone will comment whether this is necessary.)
Assuming this is necessary, you could add the PyProj library to give you a bridge between the GPS coordinates you have and the Cartesian coordinates you need. PyProj is the one-size-fits-all solution to this problem. However if UTM coordinates will work you might find the library cited here to be easier to implement.
If you decide to go with PyProj, it will help to know that your GPS data is described by the EPSG:4326 coordinate system. And if you are comfortable with UTM for your projected coordinates, you'll need need to determine an appropriate UTM zone for your area and get its Proj4 coordinate definition from SpatialReference.org.
For example I live in South Carolina, USA, which is UTM 17 North. So if I go to SpatialReference.org, search for "EPSG UTM zone 17N", select the option which references "WGS 1984" (I happen to know this means units in meters), then click on the Proj4 link, the site provides the coordinate system definition I'm after in Proj4 notation:
+proj=utm +zone=17 +ellps=WGS84 +datum=WGS84 +units=m +no_defs
If you're not comfortable diving into the world of coordinate systems, EPSG codes, Proj4 strings and such, you might want to favor that alternate coordinate translation library I mentioned earlier rather than PyProj. On the other hand, if you will benefit from a more localized coordinate system (most countries have their own localized systems), or if you need to keep your code portable for use in many areas, I'd recommend using PyProj and make sure to keep your Proj4 definition string in a config file, and NOT hard-coded throughout your app!
I am trying to properly project Latitude and Longitude into cartesian coordinates so that I can use an unscented Kalman filter to smooth out some GPS data. I am using constant jerk newtonian motion as my state transfer matrix. My issue is that I have tried many popular projections(Mercator, etc.) and when I reproject into latitude and longitude, my data is stretched with respect to maps. Any insight on my situation would be greatly appreciated! If you need more information please ask in the comments. Thank you.
The stretch factor between longitude and latitude is cos(latitude).
So you can do a local Cyclindrical Equidistant transformation.
this is a simple tranformation, but is only acurate around 10-100km
of the center (longitudeCenter, latitudCenter). this kind of projektion needs a (fixed) center.
Look further for "Cyclindrical Equidistant Projection" to get the simple formula.
Hi I would like to obtain a list of public locations (restaurant, hotels, cinema etc.) neighbours based on GPS coordinates. Is this possible with BigQuery ?
If you have lat-lon or GPS coordinates as columns, you could definitely grab rectangular regions from BigQuery using WHERE comparisons on the coordinates and then aggregate on the selected rows.
The scalar operations available in BigQuery are pretty powerful too -- you can add a variety of arithmetic functions to your query and still get excellent performance.
You find listed example queries on the linked page:
Return a collection of points within a rectangular bounding box centered around San Francisco (37.46, -122.50).
Return a collection of up to 100 points within an approximated circle determined by the using the Spherical Law of Cosines, centered around Denver Colorado (39.73, -104.98).
GCP announced new geospatial data types and functions with BigQuery GIS.New functions and data types follow the SQL/MM Spatial standard and will be familiar to PostGIS users and anyone already doing geospatial analysis in SQL.
Also, a new lightweight visualization tool called BigQuery Geo Viz is announced which is designed for BigQuery users that want to plot and style their geospatial query results on a map.
Implementation, currently in alpha. You can request to get access.
More details can be found here - GCP Blog
I want to be able to determine if a GPS location is in an inhabited or uninhabited zone.
I have tried several reverse geocoding API like Nominatim, but failed to get good results. It always returns the nearest possible address, even when I selected a location in the middle of a forest.
Is there any way to determine this with reasonable accuracy? Are there any databases or web services for this ?
If you have to calculate that youself, then the interesting things start:
The information whether or not a region is inhabited is stored in digital maps in layer "Land_Use". There are values for Forest, Water, Industry, Cemetary, etc.
You would have to import these Land_use polygons into a special DB (PostGres).
Such a spatial DB provides fast geo indizeds for searching only the relevant polygons.
Some countries may also fit in main memory, but then you need some kind of geo spatial index, like Quad-Tree or k-d tree to store the polygons.
Once you have imported the polygons, it is a simple "point in polygon" query, or "polygons within radius r". The typoe of th epolygon denotes the land use.
OpenStreetMap provides these polygons for free.
Otherwise you have to buy them from TomTom or probably NavTeq (Nokia Maps). But this makes only sense for major companies.
Since you're using Nominatim, you're getting the coordinates of the nearest address back in the reply.
Since the distance between two coordinates can be calculated, you can just use that to calculate the distance to the closest address found, and from that figure out if you're close to populated areas or not.