I have a data set that has location data of an animal. The data set has very large number of records. I expect to extract latitude and longitude data from the data set that was provided. It might be important to state that these coordinates correspond to locations in Sri Lanka. I found that longitudes in Sri Lanka range from 79.81-81.81 where latitudes range from 9.77-5.94.
X,Y,Date,Time
540564.950532,697307.393351,24.07.2012,1:30:19
541154.889569,697570.313356,24.07.2012,5:30:48
421860.001784,869773.695226,25.06.2008,20:01
513559.094669,681860.728185,2006.08.19,4:00:36
A real raw data sample can be found from [here][1]
Unfortunately, I am not in a position to ask the sender of the DMS format of x,y coordinates.
What I am after is a mechanism to convert x,y into decimal degree format. The challenge here is that it is unknown the format of the x,y string. it can be DDMMMM.SSSSSS or anything.
Related
I want to convert Dutch RD coordinates to longitude and latitude decimal degree coordinates. Example:
108519 438598
108518 438578
108517 438578
to
51.93391 4.71134
51.93382 4.71133
51.93373 4.71131
Which packages and what code can I use to apply this on a bigger dataset?
For coordinate conversion one usually uses the proj.4 lib.
Its available for many programming languages, like python, java, c
First you need to find out the projection number as EPSG number.
e.g https://epsg.io/28992
On that page under "export" there is a section for the proj.4 definition of that projection, which gives this string:
"+proj=sterea +lat_0=52.15616055555555 +lon_0=5.38763888888889 +k=0.9999079 +x_0=155000 +y_0=463000 +ellps=bessel +towgs84=565.417,50.3319,465.552,-0.398957,0.343988,-1.8774,4.0725 +units=m +no_defs"
Using the proj4 lib, you can then convert to WGS84 latitude and longitude, this is the format you want.
I am using this query to extract the geometries of all countries using BigQuery public dataset, see question here
how to extract all countries geometry from Openstreet map dataset in BigQuery
I use R to draw the results
I tried Kepler.GL and gave me the same results
Something is wrong with Russia and the USA
I know little about R visualization, but what is probably happening is you getting WKT text from BigQuery, and feeding it to R, which has different assumptions.
Tthe issue is your R package probably treats WKT differently than BigQuery. WKT semantics depends on the spatial reference system (SRS) used, which could be geographic (non-projected, using sphere or ellipsoid) or projected (flat map). BigQuery uses geographic system, so edge between points A and B is the shortest geodesic path. Most visualization systems use projected coordinates, and assume flat map. Edge between A and B is shortest straight line on the flat map.
While this does not matter too much in many cases, it still does affect precision when you have long edges. But when an edge crosses anti-meridian (180 degree meridian) you get big problem. An edge between (-169, 66) (eastern edge of Russia) and say (176, 70) (a nearby point on Chukchi sea) is relatively short on the sphere, it crosses anti-meridian, and spans 15 degrees longitude. But the same edge on flat map span 145 degrees longitude and crosses most of the map! These are the long near-horizontal lines you see.
What should you do?
If R has a packet that supports geographic SRS (it is sometime an option to use geodesic edges), you could try it.
Or you can also let BigQuery convert geography from geographic SRS to flat map, that R would understand, using ST_AsGeoJson function. GeoJson is defined on flat map, so BigQuery ST_AsGeoJson converts the semantics from geographic SRS to flat map SRS. You then visualize GeoJson string instead of WKT string in R.
ST_AsGeoJson does a lot of work, to make result conformant to GeoJson spec and flat map. It splits parts of geography that lay east and west of anti-meridian, so you don't get edges that cross it. It also approximates geodesic edges with flat map edges. But it makes visualization system much easier.
I am trying to figure out a way to convert the result of presto geo spatial function ST_DISTANCE to meters.
If I run the this example query:
SELECT ST_Distance(ST_Point(0.3476, 32.5825),ST_Point(0.0512, 32.4637))
The result I get from Presto is: 0.3193217812802629. The actual distance between these two places is 40,000m.
The presto documentation states that ST_DISTANCE: Returns the 2-dimensional cartesian minimum distance (based on spatial ref) between two geometries in projected units.
What I can understand about spatial ref is at links such as these:
http://webhelp.esri.com/arcgiSDEsktop/9.3/index.cfm?TopicName=Defining_a_spatial_reference
Which leads me to believe I need to figure you what spatial-ref Presto is using.
If I check the prest docs here:
https://github.com/prestodb/presto/blob/master/presto-geospatial/src/main/java/com/facebook/presto/geospatial/GeoFunctions.java
I can guess that is using the ESRI libraries so I assume the ESRI spatial ref? But that is where I get a bit lost as to where to proceed?
Thank you for your help..
I would recommend using Presto’s great_circle_distance() function instead of ST_Distance(). It will interpret your coordinates as WGS84 (aka EPSG:4326), and find the distance between them in kilometres by treating the shape of the earth as a sphere.
ST_Distance() would be appropriate if the coordinate system being used was already projected into a system that used metres or miles or some other unit, but there's no trivial way to do that in Presto.
From looking at the docs, it appears that presto supports a geometry type but not a geography type. That means it's not working with Latitude and Longitude, which is what I assume you're supplying as those point parameters. It's just an arbitrary 2D grid and so the resulting units are in whatever units you supplied as input.
The distance, in meters, between two points which are both approximately 32.5 meters "up" from the origin and about 0.5 meters "left" from the origin (how presto will have interpreted your points) is, indeed, 0.3193217812802629, the value that was returned to you.
How to calculate Altitude from GPS Latitude and Longitude values.What is the exact mathematical equation to solve this problem.
It is possible for a given lat,lon to determine the height of the ground (above sea level, or above Referenz Elipsoid).
But since the earth surface, mountains, etc, do not follow a mathematic formula,
there are Laser scans, performed by Satelites, that measured such a height for e.g every 30 meters.
So there exist files where you can lookup such a height.
This is called a Digital Elevation Modell, or short (DEM)
Read more here: https://en.wikipedia.org/wiki/Digital_elevation_model
Such files are huge, very few application use that approach.
Many just take the altidude value as delivered by the GPS receiver.
You can find some charts with altitude data, like Maptech's. Each pixel has a corresponding lat, long, alt/depth information.
As #AlexWien said these files are huge and most of them must be bought.
If you are interest of using these files I can help you with a C++ code to read them.
You can calculate the geocentric radius, i.e., the radius of the reference Ellipsoid which is used as basis for the GPS altitude. It can be calculated from the the GPS latitude with this formula:
Read more about this at Wikipedia.
I'm working with a postgresql table that contains many rows with a GEOMETRY(Point, 4326). Using the ST_SnapToGrid function and a DISTINCT select, I only extract a subset of rows depending on the displayed map zoom level. I'm having trouble finetuning the ST_SnapToGrid function, as I don't not what unit the size parameter is in?
The size is specified in the same units as the SRID of the geometry.
In the case of SRID 4326 this is decimal degrees. The actual distance
that is of course varies depending on the actual latitude and longitude
of the point in question. If this matters, if might help to work in
some projected coordinates and do the rounding there.