I am using this query to extract the geometries of all countries using BigQuery public dataset, see question here
how to extract all countries geometry from Openstreet map dataset in BigQuery
I use R to draw the results
I tried Kepler.GL and gave me the same results
Something is wrong with Russia and the USA
I know little about R visualization, but what is probably happening is you getting WKT text from BigQuery, and feeding it to R, which has different assumptions.
Tthe issue is your R package probably treats WKT differently than BigQuery. WKT semantics depends on the spatial reference system (SRS) used, which could be geographic (non-projected, using sphere or ellipsoid) or projected (flat map). BigQuery uses geographic system, so edge between points A and B is the shortest geodesic path. Most visualization systems use projected coordinates, and assume flat map. Edge between A and B is shortest straight line on the flat map.
While this does not matter too much in many cases, it still does affect precision when you have long edges. But when an edge crosses anti-meridian (180 degree meridian) you get big problem. An edge between (-169, 66) (eastern edge of Russia) and say (176, 70) (a nearby point on Chukchi sea) is relatively short on the sphere, it crosses anti-meridian, and spans 15 degrees longitude. But the same edge on flat map span 145 degrees longitude and crosses most of the map! These are the long near-horizontal lines you see.
What should you do?
If R has a packet that supports geographic SRS (it is sometime an option to use geodesic edges), you could try it.
Or you can also let BigQuery convert geography from geographic SRS to flat map, that R would understand, using ST_AsGeoJson function. GeoJson is defined on flat map, so BigQuery ST_AsGeoJson converts the semantics from geographic SRS to flat map SRS. You then visualize GeoJson string instead of WKT string in R.
ST_AsGeoJson does a lot of work, to make result conformant to GeoJson spec and flat map. It splits parts of geography that lay east and west of anti-meridian, so you don't get edges that cross it. It also approximates geodesic edges with flat map edges. But it makes visualization system much easier.
Related
I have 20 000 polygons in a dataset. I need to have the Euclidean Distance between all polygons, so a 20 000 x 20 000 distance matrix where for each of the polygons, the distance to all other polygons is stored.
I have read in some other threads the recommendation to use the "Near" tool in Arcmap. However, this tool only calculates the distance to the NEAREST polygon, while I need the distance from ALL polygons to ALL polygons.
Is there any solution for this?
Near tool: Calculates distance and additional proximity information between the
input features and the closest feature in another layer or feature
class.
In order to calculate the distance between the centroids of each of your polygons make sure your map is in a projected coordinate system.
Then, make sure the centroid points are calculated (detailed step-by-step here: https://support.esri.com/en/technical-article/000009381 )
Export your centroid point attribute table as a DBF (Click on Options > Export.)
Add the table to your map. Right click on the new table, Display XY Data, select Longitude for the X and Latitude for Y, and select the map's coordinate system to create an events layer.
Then, use the Point Distance tool (Details here: https://desktop.arcgis.com/en/arcmap/10.3/tools/analysis-toolbox/point-distance.htm ). The event points are both the input and near features. The output will be a table displaying distance between all polygon centroids on the map.
I am trying to figure out a way to convert the result of presto geo spatial function ST_DISTANCE to meters.
If I run the this example query:
SELECT ST_Distance(ST_Point(0.3476, 32.5825),ST_Point(0.0512, 32.4637))
The result I get from Presto is: 0.3193217812802629. The actual distance between these two places is 40,000m.
The presto documentation states that ST_DISTANCE: Returns the 2-dimensional cartesian minimum distance (based on spatial ref) between two geometries in projected units.
What I can understand about spatial ref is at links such as these:
http://webhelp.esri.com/arcgiSDEsktop/9.3/index.cfm?TopicName=Defining_a_spatial_reference
Which leads me to believe I need to figure you what spatial-ref Presto is using.
If I check the prest docs here:
https://github.com/prestodb/presto/blob/master/presto-geospatial/src/main/java/com/facebook/presto/geospatial/GeoFunctions.java
I can guess that is using the ESRI libraries so I assume the ESRI spatial ref? But that is where I get a bit lost as to where to proceed?
Thank you for your help..
I would recommend using Presto’s great_circle_distance() function instead of ST_Distance(). It will interpret your coordinates as WGS84 (aka EPSG:4326), and find the distance between them in kilometres by treating the shape of the earth as a sphere.
ST_Distance() would be appropriate if the coordinate system being used was already projected into a system that used metres or miles or some other unit, but there's no trivial way to do that in Presto.
From looking at the docs, it appears that presto supports a geometry type but not a geography type. That means it's not working with Latitude and Longitude, which is what I assume you're supplying as those point parameters. It's just an arbitrary 2D grid and so the resulting units are in whatever units you supplied as input.
The distance, in meters, between two points which are both approximately 32.5 meters "up" from the origin and about 0.5 meters "left" from the origin (how presto will have interpreted your points) is, indeed, 0.3193217812802629, the value that was returned to you.
I have a point cloud (data set) (3D) representing an urban terrain consisting of flat roof surfaces (of buildings) . My aim is to figure out the flat surfaces , waterbodies from the given data set .The data set is a text file consisting of the number of points followed by their individual x , y , z co-ordinates. As a trial attempt , I have generated the 2D-Delaunay triangulation of the given data set to get the triangulated surface. Henceforth, I plan to execute a graph-traversal over the faces of the triangulation to look for neighbourhood points with nearly the same z-coordinate value and treat them as a flat surface . I am using CGAL libraries to accomplish the same in C++. Is there a better approach for identifying flat surface or my method seems decent enough ?
You might get inspiration (or just run) Advancing Front Reconstruction
No idea how your point cloud looks, but maybe Point Set Shape Detection
might help you to identify flat areas.
I'm having some spatial data that has all of its coordinates as lat/lon pairs (with about 10 digits decimal precision), it's stored in a database as WGS84 data.Some of the data is represented as polygons which are the resulting union of some smaller polygons whose boundaries are stored.Then I'm having a number of points from which I build a linesegments (just 2 points in each segment) which I use later for intersection tests with the polygons.
I'm using a SpatialIndex to improve my queries so I insert the envelopes of all polygons in a tree (tested with both QuadTree and STRtree).Then, I connect two points into a linesegment and I'm using its envelope to query the tree for possible intersections.The problem is that I get pretty much all the polygons as a result which is clearly wrong.. To give you some idea about the real scale of my data, I have about 100 polygons that cover the whole North america, each line covers a very very small part of a single polygon.Ideally, i would expect no more than 2 polygons as a result.
I'm using JTS to do this calculation and I'm aware that it's not really suited for spherical data so can you suggest me another library/tool to achieve the desired behaviour or possible a workaround (for example, projecting before using JTS)?
If you only have north america, just rotate earth by 90 degrees so that Alaska is no longer on the far east. (Fun fact: Alaska is both the most northern, western and eastern state of the U.S.) Then your rectangles should be okay.
There are a number of non-trivial cases though when working with spherical data. Depending on how your data is defined, your polygon borders may actually be bent lines, instead of straight lines. Consider this screenshot of Google Ingress: https://lh4.ggpht.com/S_9jrMqf08JfIbr7DgUDH96rvXMK4wOGtaSKYPGCruXv2HE4oeRuEaQIDIywMgH4198=h900
I read somewhere that the mismatch of the "fog" texture and the green line visible in the left field is due to the two drawing functions using different approximations. One is always a straight line, whereas the other follows the curvature of the earth. If you have a large field (polygon!), the error becomes worse.
"Intersection" becomes a tricky term when your data consists of non-straight lines on the surface of a sphere, unfortunately; and a "straight" line on the surface of earth will often yield an arctan type curve in latlon coordinates.
Projections: these can help, but mostly when your data is local. UTM projections are pretty good, but you need at least 9 UTM zones to cover north america without Alaska. As long as your data is within one UTM zone, projecting the data into this zone and then working with 2D euclidean space should work good. But if it gets lager than this, you may need to stitch different projections, and that is really messy, too.
as topic, the Coordinates value (Latitude and Longitude) is known , these Coordinates will compose as polygonal area , my question is how to calculate the area of the polygonal that is base the geography ?
thanks for your help .
First you would need to know whether the curvature of the surface would be significant. If it is a relatively small then you can get a good approximation by projecting the coordinates onto a plane.
Determine units of measure per degree of latitude (eg. meters per degree)
Determine units of meature per degree of longitude at a given latitude (the conversion factor varies as you go North or South)
Convert latitude and longitude pairs to (x,y) pairs in the plane
Use an algorithm to compute area of a polygon. See StackOverflow 451425 or Paul Bourke
If you are calculating a large area then spherical techniques must be used.
If I understand your question correctly - triangulation should help you. Basically you break the polygonal to triangles in such a way that they don't overlap and sum their areas.