Oracle Spatial Objects to Vertex List - sql
I am struggling to convert a series of oracle SDOs (polygons specifically) into a more usable format.
I have data that is in this format:
PolygonID | Polygon
1 | SDO Geometry
2 | SDO Geometry
3 | SDO Geometry
And so on...
What i want to get is the following:
PolygonID | Vertex.X | Vertex.Y | Vertex.Order
1 | 1 | 1 | 1
1 | 3 | 5 | 2
1 | 2 | 3 | 3
2 | 1 | 2 | 1
And so on. So I just need to polygon converted into a ordered list of vertices. I can successfully convert a single SDO geometry into a ordered list using the below code but cant link it to its polygon ID.
select x,y
from table (
select sdo_util.getvertices(
SDO
)
from POLYGONS
where ID = 1
)
order by id;
I am a bit lost on how to link that data back to its original polygon ID. Any help would be greatly appreciated!
So I eventually found the solution. See below.
SELECT
c.POLYGONID,
t.X,
t.Y,
t.ID as ORDER
FROM POLYGONS c,
TABLE(SDO_UTIL.GETVERTICES(c.POLYGON)) t
The table function creates a sub-query linked to each polygon ID.
Related
Minimizing a graph with SQL
Suppose we have a directed graph defined as following: node | neighbor ----------------- 1 | 2 1 | 3 2 | 4 2 | 3 3 | 4 the above table defines the only the edges between two nodes, a couple (1,2)for example means that node 1 and 2 are connected by an edge, here is a plot of the graph. I also have a table of the transitive closure of the graph, this table holds all the possible paths of the graph (for example: (1,3) is present twice because it can be reached either directly or by the path 1=>2=>3), here is the table: node | neighbor ----------------- 1 | 2 1 | 3 2 | 4 2 | 3 3 | 4 1 | 3 1 | 4 1 | 4 2 | 4 from these two tables, I want to return a minimized graph without losing any reachability, an idea was to only return edges that are not in dependency of the two tables, here's an example: (1,2) is in the first table and (2,3) is in the second, and therefore (1,3) can be deleted from the first table because you can reach node 3 from 1 passing by node 2 the outuput table should look like this then: node | neighbor ----------------- 1 | 2 2 | 3 3 | 4 How can I write an SQL query that does this?
Here is one approach: with recursive cte as ( select node, neighbor, 1 is_initial from graph union all select c.node, g.neighbor, 0 from cte c inner join graph g on g.node = c.neighbor ) select node, neighbor from graph g where not exists ( select 1 from cte c where c.node = g.node and c.neighbor = g.neighbor and c.is_initial = 0 ) order by node, neighbor This uses the first table only (I called it graph). We start by generating all possible paths with a recursive query. This is quite similar to your closure table, but with one extra column, is_initial, that indicates whether the path comes from the original table, or was generated during a further iteration. Then, all that is left to do is filter the graph to remove tuples that match a "non-initial" path. Demo on DB Fiddle: node | neighbor ---: | -------: 1 | 2 2 | 3 3 | 4
Recursive self join over file data
I know there are many questions about recursive self joins, but they're mostly in a hierarchical data structure as follows: ID | Value | Parent id ----------------------------- But I was wondering if there was a way to do this in a specific case that I have where I don't necessarily have a parent id. My data will look like this when I initially load the file. ID | Line | ------------------------- 1 | 3,Formula,1,2,3,4,... 2 | *,record,abc,efg,hij,... 3 | ,,1,x,y,z,... 4 | ,,2,q,r,s,... 5 | 3,Formula,5,6,7,8,... 6 | *,record,lmn,opq,rst,... 7 | ,,1,t,u,v,... 8 | ,,2,l,m,n,... Essentially, its a CSV file where each row in the table is a line in the file. Lines 1 and 5 identify an object header and lines 3, 4, 7, and 8 identify the rows belonging to the object. The object header lines can have only 40 attributes which is why the object is broken up across multiple sections in the CSV file. What I'd like to do is take the table, separate out the record # column, and join it with itself multiple times so it achieves something like this: ID | Line | ------------------------- 1 | 3,Formula,1,2,3,4,5,6,7,8,... 2 | *,record,abc,efg,hij,lmn,opq,rst 3 | ,,1,x,y,z,t,u,v,... 4 | ,,2,q,r,s,l,m,n,... I know its probably possible, I'm just not sure where to start. My initial idea was to create a view that separates out the first and second columns in a view, and use the view as a way of joining in a repeated fashion on those two columns. However, I have some problems: I don't know how many sections will occur in the file for the same object The file can contain other objects as well so joining on the first two columns would be problematic if you have something like ID | Line | ------------------------- 1 | 3,Formula,1,2,3,4,... 2 | *,record,abc,efg,hij,... 3 | ,,1,x,y,z,... 4 | ,,2,q,r,s,... 5 | 3,Formula,5,6,7,8,... 6 | *,record,lmn,opq,rst,... 7 | ,,1,t,u,v,... 8 | ,,2,l,m,n,... 9 | ,4,Data,1,2,3,4,... 10 | *,record,lmn,opq,rst,... 11 | ,,1,t,u,v,... In the above case, my plan could join rows from the Data object in row 9 with the first rows of the Formula object by matching the record value of 1. UPDATE I know this is somewhat confusing. I tried doing this with C# a while back, but I had to basically write a recursive decent parser to parse the specific file format and it simply took to long because I had to get it in the database afterwards and it was too much for entity framework. It was taking hours just to convert one file since these files are excessively large. Either way, #Nolan Shang has the closest result to what I want. The only difference is this (sorry for the bad formatting): +----+------------+------------------------------------------+-----------------------+ | ID | header | x | value | +----+------------+------------------------------------------+-----------------------+ | 1 | 3,Formula, | ,1,2,3,4,5,6,7,8 |3,Formula,1,2,3,4,5,6,7,8 | | 2 | ,, | ,1,x,y,z,t,u,v | ,1,x,y,z,t,u,v | | 3 | ,, | ,2,q,r,s,l,m,n | ,2,q,r,s,l,m,n | | 4 | *,record, | ,abc,efg,hij,lmn,opq,rst |*,record,abc,efg,hij,lmn,opq,rst | | 5 | ,4, | ,Data,1,2,3,4 |,4,Data,1,2,3,4 | | 6 | *,record, | ,lmn,opq,rst | ,lmn,opq,rst | | 7 | ,, | ,1,t,u,v | ,1,t,u,v | +----+------------+------------------------------------------+-----------------------------------------------+
I agree that it would be better to export this to a scripting language and do it there. This will be a lot of work in TSQL. You've intimated that there are other possible scenarios you haven't shown, so I obviously can't give a comprehensive solution. I'm guessing this isn't something you need to do quickly on a repeated basis. More of a one-time transformation, so performance isn't an issue. One approach would be to do a LEFT JOIN to a hard-coded table of the possible identifying sub-strings like: 3,Formula, *,record, ,,1, ,,2, ,4,Data, Looks like it pretty much has to be human-selected and hard-coded because I can't find a reliable pattern that can be used to SELECT only these sub-strings. Then you SELECT from this artificially-created table (or derived table, or CTE) and LEFT JOIN to your actual table with a LIKE to get all the rows that use each of these values as their starting substring, strip out the starting characters to get the rest of the string, and use the STUFF..FOR XML trick to build the desired Line. How you get the ID column depends on what you want, for instance in your second example, I don't know what ID you want for the ,4,Data,... line. Do you want 5 because that's the next number in the results, or do you want 9 because that's the ID of the first occurrance of that sub-string? Code accordingly. If you want 5 it's a ROW_NUMBER(). If you want 9, you can add an ID column to the artificial table you created at the start of this approach. BTW, there's really nothing recursive about what you need done, so if you're still thinking in those terms, now would be a good time to stop. This is more of a "Group Concatenation" problem.
Here is a sample, but has some different with you need.
It is because I use the value the second comma as group header, so the ,,1 and ,,2 will be treated as same group, if you can use a parent id to indicated a group will be better
DECLARE #testdata TABLE(ID int,Line varchar(8000))
INSERT INTO #testdata
SELECT 1,'3,Formula,1,2,3,4,...' UNION ALL
SELECT 2,'*,record,abc,efg,hij,...' UNION ALL
SELECT 3,',,1,x,y,z,...' UNION ALL
SELECT 4,',,2,q,r,s,...' UNION ALL
SELECT 5,'3,Formula,5,6,7,8,...' UNION ALL
SELECT 6,'*,record,lmn,opq,rst,...' UNION ALL
SELECT 7,',,1,t,u,v,...' UNION ALL
SELECT 8,',,2,l,m,n,...' UNION ALL
SELECT 9,',4,Data,1,2,3,4,...' UNION ALL
SELECT 10,'*,record,lmn,opq,rst,...' UNION ALL
SELECT 11,',,1,t,u,v,...'
;WITH t AS(
SELECT *,REPLACE(SUBSTRING(t.Line,LEN(c.header)+1,LEN(t.Line)),',...','') AS data
FROM #testdata AS t
CROSS APPLY(VALUES(LEFT(t.Line,CHARINDEX(',',t.Line, CHARINDEX(',',t.Line)+1 )))) c(header)
)
SELECT MIN(ID) AS ID,t.header,c.x,t.header+STUFF(c.x,1,1,'') AS value
FROM t
OUTER APPLY(SELECT ','+tb.data FROM t AS tb WHERE tb.header=t.header FOR XML PATH('') ) c(x)
GROUP BY t.header,c.x
+----+------------+------------------------------------------+-----------------------------------------------+
| ID | header | x | value |
+----+------------+------------------------------------------+-----------------------------------------------+
| 1 | 3,Formula, | ,1,2,3,4,5,6,7,8 | 3,Formula,1,2,3,4,5,6,7,8 |
| 3 | ,, | ,1,x,y,z,2,q,r,s,1,t,u,v,2,l,m,n,1,t,u,v | ,,1,x,y,z,2,q,r,s,1,t,u,v,2,l,m,n,1,t,u,v |
| 2 | *,record, | ,abc,efg,hij,lmn,opq,rst,lmn,opq,rst | *,record,abc,efg,hij,lmn,opq,rst,lmn,opq,rst |
| 9 | ,4, | ,Data,1,2,3,4 | ,4,Data,1,2,3,4 |
+----+------------+------------------------------------------+-----------------------------------------------+
Postgres/Postgis - Querying all values in clipped raster
I'm currently using Postgres 9.1 My goal is to clip a PostGIS raster with a polygon. Then I would like either an postgres array or delimited set of values for each of the raster pixels that are contained within that polygon. Here is the query I have gotten so far: SELECT p.gid, (ST_ValueCount(ST_Clip(r.rast, p.geom))).value AS fval FROM temp_raster AS r, temp_shapefile AS p WHERE (r.start_time = 1384516800) GROUP BY gid, fval; This will output: gid | fcstval -----+--------- 1 | 0 1 | 2 2 | 0 2 | 2 3 | 5 4 | 0 4 | 1 4 | 2 4 | 3 4 | 5 This data is good, but I would like a set of values for each gid like this: gid | fcstval -----+---------- 1 | 0,2 2 | 0,2 3 | 5 4 | 0,1,2,3,5 The trouble I'm having is trying to then get these values aggregated into either an array or delimited string. Here's my attempt at an array: SELECT p.gid, array_agg((ST_ValueCount(ST_Clip(r.rast, p.geom))).value) AS fval FROM temp_raster AS r, temp_shapefile AS p WHERE (r.start_time = 1384516800) GROUP BY gid; This doesn't work, and provides the error: ERROR: set-valued function called in context that cannot accept a set My guess is this is because I can't call array_agg in this way. I'm having a bit of difficulty figuring out how to do this otherwise. A subquery perhaps? I haven't been able to come up with anything yet though to get this working. Thanks for any help!
Okay I think I figured it out for arrays. If I want it in a string, I can just convert my array to a string. However if anyone has any suggestions of cleaning this up I would appreciate it, as this doesn't seem like the simplest markup
SELECT p.gid, array_agg((subquery).tempval) as fcstval
FROM pih_raster AS r, hwy_pih_vertex_buf AS p,
(
SELECT p.gid AS tempgid, (ST_ValueCount(ST_Clip(r.rast, p.geom))).value AS tempval
FROM pih_raster AS r, hwy_pih_vertex_buf AS p
WHERE (r.start_time <= 1384624800) GROUP BY tempgid, tempval
) AS subquery
WHERE (r.start_time <= 1384624800) AND ((subquery).tempgid = gid) GROUP BY p.gid;
Here's the output:
gid | fcstval
-----+-------------
1 | {0,2}
2 | {0,2}
3 | {5}
4 | {0,1,2,3,5}
How do I do a WHERE NOT IN for Hierarchical data?
I have a table that is a list of paths between points. I want to create a query to return a list with pointID and range(number of point) from a given point. But have spent a day trying to figure this out and haven't go any where, does any one know how this should be done? ( I am writing this for MS-SQL 2005) example -- fromPointID | toPointID | ---------------|-----------| -- 1 | 2 | -- 2 | 1 | -- 1 | 3 | -- 3 | 1 | -- 2 | 3 | -- 3 | 2 | -- 4 | 2 | -- 2 | 4 | with PointRanges ([fromPointID], [toPointID], [Range]) AS ( -- anchor member SELECT [fromPointID], [toPointID], 0 AS [Range] FROM dbo.[Paths] WHERE [toPointID] = 1 UNION ALL -- recursive members SELECT P.[fromPointID], P.[toPointID], [Range] + 1 AS [Range] FROM dbo.[Paths] AS P INNER JOIN PointRanges AS PR ON PR.[toPointID] = P.[fromPointID] WHERE [Range] < 5 -- This part is just added to limit the size of the table being returned --WHERE P.[fromPointID] NOT IN (SELECT [toPointID] FROM PointRanges) --Cant do the where statment I want to because it wont allow recurssion in the sub query ) SELECT * FROM PointRanges --Want this returned -- PointID | Range | -----------|-------| -- 1 | 0 | -- 2 | 1 | -- 3 | 1 | -- 4 | 2 |
Markus Jarderot's link gives a good answer for this. I end tried using his answer and also tried rewriting my problem in C# and linq but it ended up being more of a mathematical problem than a coding problem because I had a table with several thousands of point that interlinked. This is still something I am interested in and trying to get a better understanding of by reading books on mathematics and graph theory but if anyone else runs into this problem I think Markus Jarderot's link is the best answer you will find.
How to properly group SQL results set?
SQL noob, please bear with me!!
I am storing a 3-tuple in a database (x,y, {signal1, signal2,..}).
I have a database with tables coordinates (x,y) and another table called signals (signal, coordinate_id, group) which stores the individual signal values. There can be several signals at the same coordinate.
The group is just an abitrary integer which marks the entries in the signal table as belonging to the same set (provided they belong to the same coordinate). So that any signals with the same 'coordinate_id' and 'group' together form a tuple as shown above.
For example,
Coordinates table Signals table
-------------------- -----------------------------
| id | x | y | | id | signal | coordinate_id | group |
| 1 | 1 | 2 | | 1 | 45 | 1 | 1 |
| 2 | 2 | 5 | | 2 | 95 | 1 | 1 |
| 3 | 33 | 1 | 1 |
| 4 | 65 | 1 | 2 |
| 5 | 57 | 1 | 2 |
| 6 | 63 | 2 | 1 |
This would produce the tuples (1,2 {45,95,33}), (1,2,{65,57}), (2,5, {63}) and so on.
I would like to retrieve the sets of {signal1, signal2,...} for each coordinate. The signals belonging to a set have the same coordinate_id and group, but I do not necessarily know the group value. I only know that if the group value is the same for a particular coordinate_id, then all those with that group form one set.
I tried looking into SQL GROUP BY, but I realized that it is for use with aggregate functions.
Can someone point out how to do this properly in SQL or give tips for improving my database structure.
SQLite supports the GROUP_CONCAT() aggregate function similar to MySQL. It rolls up a set of values in the group and concatenates them together comma-separated. SELECT c.x, c.y, GROUP_CONCAT(s.signal) AS signal_list FROM Signals s JOIN Coordinates ON s.coordinate_id = c.id GROUP BY s.coordinate_id, s.group SQLite also permits the mismatch between columns in the select-list and columns in the group-by clause, even though this isn't strictly permitted by ANSI SQL and most implementations.
personally I would write the database as 3 tables: x_y(x, y, id) coords_groups(pos, group, id) signals(group, signal) with signals.group->coords_groups.id and coords_groups.pos->x_y.id as you are trying to represent a sort-of 4 dimensional array. then, to get from a couple of coordinates (X, Y) an ArrayList of List of Signal you can use this SELECT temp."group", signals.signal FROM ( SELECT cg."group", cg.id FROM x_y JOIN coords_groups AS cg ON x_y.id = cg.pos WHERE x_y.x=X AND x_y.y=Y ) AS temp JOIN signals ON temp.id=signals."group" ORDER BY temp."group" ASC (X Y are in the innermost where) inside this sort of pseudo-code: getSignalsGroups(X, Y) ArrayList<List<Signals>> a List<Signals> temp query=sqlLiteExecute(THE_SQL_SNIPPET, x, y) row=query.fetch() //fetch the first row to set the groupCounter actualGroup=row.group temp.add(row.signal) for(row : query) //foreach row add the signal to the list if(row.group!=actualGroup) //or reset the list if is a new group a.add(actualGroup, temp) actualGroup=row.group; temp= new List temp.add(row.signal) return a