Postgres/Postgis - Querying all values in clipped raster - sql

I'm currently using Postgres 9.1
My goal is to clip a PostGIS raster with a polygon. Then I would like either an postgres array or delimited set of values for each of the raster pixels that are contained within that polygon. Here is the query I have gotten so far:
SELECT p.gid, (ST_ValueCount(ST_Clip(r.rast, p.geom))).value AS fval
FROM temp_raster AS r, temp_shapefile AS p
WHERE (r.start_time = 1384516800)
GROUP BY gid, fval;
This will output:
gid | fcstval
-----+---------
1 | 0
1 | 2
2 | 0
2 | 2
3 | 5
4 | 0
4 | 1
4 | 2
4 | 3
4 | 5
This data is good, but I would like a set of values for each gid like this:
gid | fcstval
-----+----------
1 | 0,2
2 | 0,2
3 | 5
4 | 0,1,2,3,5
The trouble I'm having is trying to then get these values aggregated into either an array or delimited string. Here's my attempt at an array:
SELECT p.gid, array_agg((ST_ValueCount(ST_Clip(r.rast, p.geom))).value) AS fval
FROM temp_raster AS r, temp_shapefile AS p
WHERE (r.start_time = 1384516800)
GROUP BY gid;
This doesn't work, and provides the error:
ERROR: set-valued function called in context that cannot accept a set
My guess is this is because I can't call array_agg in this way. I'm having a bit of difficulty figuring out how to do this otherwise. A subquery perhaps? I haven't been able to come up with anything yet though to get this working.
Thanks for any help!

Okay I think I figured it out for arrays. If I want it in a string, I can just convert my array to a string. However if anyone has any suggestions of cleaning this up I would appreciate it, as this doesn't seem like the simplest markup
SELECT p.gid, array_agg((subquery).tempval) as fcstval
FROM pih_raster AS r, hwy_pih_vertex_buf AS p,
(
SELECT p.gid AS tempgid, (ST_ValueCount(ST_Clip(r.rast, p.geom))).value AS tempval
FROM pih_raster AS r, hwy_pih_vertex_buf AS p
WHERE (r.start_time <= 1384624800) GROUP BY tempgid, tempval
) AS subquery
WHERE (r.start_time <= 1384624800) AND ((subquery).tempgid = gid) GROUP BY p.gid;
Here's the output:
gid | fcstval
-----+-------------
1 | {0,2}
2 | {0,2}
3 | {5}
4 | {0,1,2,3,5}

Related

Reference the output of a calculated column in Hive SQL

I have a self-referencing/recursive calculation in Excel that needs to be moved to Hive SQL. Basically the column needs to SUM the two values only if the total of the concrete column plus the result from the previous calculation is greater than 0.
The data is as follows, A is the value and B is the expected output:
| A | B |
|-----|-----|
| -1 | 0 |
| 2 | 2 |
| -2 | 0 |
| 2 | 2 |
| 2 | 4 |
| -1 | 3 |
| 2 | 5 |
In Excel it would be written in column B as:
=MAX(0,B1+A2)
The problem in SQL is you need to have the output of the current calculation. I think I've got it sorted in SQL as the following:
DECLARE #Numbers TABLE(A INT, Rn INT)
INSERT INTO #Numbers VALUES (-1,1),(2,2),(-2,3),(2,4),(2,5),(-1,6),(2,7);
WITH lagged AS
(
SELECT A, 0 AS B, Rn
FROM #Numbers
WHERE Rn = 1
UNION ALL
SELECT i.A,
CASE WHEN ((i.A + l.B) >= 0) THEN (i.A + l.B)
ELSE l.B
END,
i.Rn
FROM #Numbers i INNER JOIN lagged l
ON i.Rn = l.Rn + 1
)
SELECT *
FROM lagged;
But this being Hive, it doesn't support CTEs so I need to dumb the SQL down a touch. Is that possible using LAG/LEAD? My brain is hurting having got this far!
I initially thought that it would help to first compute the Sum of all elements until each rank and then fix the values somehow using negative elements.
However, one big negative that would zero the B column will carry forward in the sum and will make all following elements negative.
It's as Gordon commented - 0 is max in the calculation =MAX(0,B1+A2) depends on the previous location where it happened and it seems to be impossible to compute them in advance analytically.

Oracle Spatial Objects to Vertex List

I am struggling to convert a series of oracle SDOs (polygons specifically) into a more usable format.
I have data that is in this format:
PolygonID | Polygon
1 | SDO Geometry
2 | SDO Geometry
3 | SDO Geometry
And so on...
What i want to get is the following:
PolygonID | Vertex.X | Vertex.Y | Vertex.Order
1 | 1 | 1 | 1
1 | 3 | 5 | 2
1 | 2 | 3 | 3
2 | 1 | 2 | 1
And so on. So I just need to polygon converted into a ordered list of vertices. I can successfully convert a single SDO geometry into a ordered list using the below code but cant link it to its polygon ID.
select x,y
from table (
select sdo_util.getvertices(
SDO
)
from POLYGONS
where ID = 1
)
order by id;
I am a bit lost on how to link that data back to its original polygon ID. Any help would be greatly appreciated!
So I eventually found the solution. See below.
SELECT
c.POLYGONID,
t.X,
t.Y,
t.ID as ORDER
FROM POLYGONS c,
TABLE(SDO_UTIL.GETVERTICES(c.POLYGON)) t
The table function creates a sub-query linked to each polygon ID.

How do I do a WHERE NOT IN for Hierarchical data?

I have a table that is a list of paths between points. I want to create a query to return a list with pointID and range(number of point) from a given point. But have spent a day trying to figure this out and haven't go any where, does any one know how this should be done? ( I am writing this for MS-SQL 2005)
example
-- fromPointID | toPointID |
---------------|-----------|
-- 1 | 2 |
-- 2 | 1 |
-- 1 | 3 |
-- 3 | 1 |
-- 2 | 3 |
-- 3 | 2 |
-- 4 | 2 |
-- 2 | 4 |
with PointRanges ([fromPointID], [toPointID], [Range])
AS
(
-- anchor member
SELECT [fromPointID],
[toPointID],
0 AS [Range]
FROM dbo.[Paths]
WHERE [toPointID] = 1
UNION ALL
-- recursive members
SELECT P.[fromPointID],
P.[toPointID],
[Range] + 1 AS [Range]
FROM dbo.[Paths] AS P
INNER JOIN PointRanges AS PR ON PR.[toPointID] = P.[fromPointID]
WHERE [Range] < 5 -- This part is just added to limit the size of the table being returned
--WHERE P.[fromPointID] NOT IN (SELECT [toPointID] FROM PointRanges)
--Cant do the where statment I want to because it wont allow recurssion in the sub query
)
SELECT * FROM PointRanges
--Want this returned
-- PointID | Range |
-----------|-------|
-- 1 | 0 |
-- 2 | 1 |
-- 3 | 1 |
-- 4 | 2 |
Markus Jarderot's link gives a good answer for this. I end tried using his answer and also tried rewriting my problem in C# and linq but it ended up being more of a mathematical problem than a coding problem because I had a table with several thousands of point that interlinked. This is still something I am interested in and trying to get a better understanding of by reading books on mathematics and graph theory but if anyone else runs into this problem I think Markus Jarderot's link is the best answer you will find.

Finding the difference between two sets of data from the same table

My data looks like:
run | line | checksum | group
-----------------------------
1 | 3 | 123 | 1
1 | 7 | 123 | 1
1 | 4 | 123 | 2
1 | 5 | 124 | 2
2 | 3 | 123 | 1
2 | 7 | 123 | 1
2 | 4 | 124 | 2
2 | 4 | 124 | 2
and I need a query that returns me the new entries in run 2
run | line | checksum | group
-----------------------------
2 | 4 | 124 | 2
2 | 4 | 124 | 2
I tried several things, but I never got to a satisfying answer.
In this case I'm using H2, but of course I'm interested in a general explanation that would help me to wrap my head around the concept.
EDIT:
OK, it's my first post here so please forgive if I didn't state the question precisely enough.
Basically given two run values (r1, r2, with r2 > r1) I want to determine which rows having row = r2 have a different line, checksum or group from any row where row = r1.
select * from yourtable
where run = 2 and checksum = (select max(checksum)
from yourtable)
Assuming your last run will have the higher run value than others, below SQL will help
select * from table1 t1
where t1.run in
(select max(t2.run) table1 t2)
Update:
Above SQL may not give you the right rows because your requirement is not so clear. But the overall idea is to fetch the rows based on the latest run parameters.
SELECT line, checksum, group
FROM TableX
WHERE run = 2
EXCEPT
SELECT line, checksum, group
FROM TableX
WHERE run = 1
or (with slightly different result):
SELECT *
FROM TableX x
WHERE run = 2
AND NOT EXISTS
( SELECT *
FROM TableX x2
WHERE run = 1
AND x2.line = x.line
AND x2.checksum = x.checksum
AND x2.group = x.group
)
A slightly different approach:
select min(run) run, line, checksum, group
from mytable
where run in (1,2)
group by line, checksum, group
having count(*)=1 and min(run)=2
Incidentally, I assume that the "group" column in your table isn't actually called group - this is a reserved word in SQL and would need to be enclosed in double quotes (or backticks or square brackets, depending on which RDBMS you are using).

How to properly group SQL results set?

SQL noob, please bear with me!!
I am storing a 3-tuple in a database (x,y, {signal1, signal2,..}).
I have a database with tables coordinates (x,y) and another table called signals (signal, coordinate_id, group) which stores the individual signal values. There can be several signals at the same coordinate.
The group is just an abitrary integer which marks the entries in the signal table as belonging to the same set (provided they belong to the same coordinate). So that any signals with the same 'coordinate_id' and 'group' together form a tuple as shown above.
For example,
Coordinates table Signals table
-------------------- -----------------------------
| id | x | y | | id | signal | coordinate_id | group |
| 1 | 1 | 2 | | 1 | 45 | 1 | 1 |
| 2 | 2 | 5 | | 2 | 95 | 1 | 1 |
| 3 | 33 | 1 | 1 |
| 4 | 65 | 1 | 2 |
| 5 | 57 | 1 | 2 |
| 6 | 63 | 2 | 1 |
This would produce the tuples (1,2 {45,95,33}), (1,2,{65,57}), (2,5, {63}) and so on.
I would like to retrieve the sets of {signal1, signal2,...} for each coordinate. The signals belonging to a set have the same coordinate_id and group, but I do not necessarily know the group value. I only know that if the group value is the same for a particular coordinate_id, then all those with that group form one set.
I tried looking into SQL GROUP BY, but I realized that it is for use with aggregate functions.
Can someone point out how to do this properly in SQL or give tips for improving my database structure.
SQLite supports the GROUP_CONCAT() aggregate function similar to MySQL. It rolls up a set of values in the group and concatenates them together comma-separated.
SELECT c.x, c.y, GROUP_CONCAT(s.signal) AS signal_list
FROM Signals s
JOIN Coordinates ON s.coordinate_id = c.id
GROUP BY s.coordinate_id, s.group
SQLite also permits the mismatch between columns in the select-list and columns in the group-by clause, even though this isn't strictly permitted by ANSI SQL and most implementations.
personally I would write the database as 3 tables:
x_y(x, y, id) coords_groups(pos, group, id) signals(group, signal)
with signals.group->coords_groups.id and coords_groups.pos->x_y.id
as you are trying to represent a sort-of 4 dimensional array.
then, to get from a couple of coordinates (X, Y) an ArrayList of List of Signal you can use this
SELECT temp."group", signals.signal
FROM (
SELECT cg."group", cg.id
FROM x_y JOIN coords_groups AS cg ON x_y.id = cg.pos
WHERE x_y.x=X AND x_y.y=Y )
AS temp JOIN signals ON temp.id=signals."group"
ORDER BY temp."group" ASC
(X Y are in the innermost where)
inside this sort of pseudo-code:
getSignalsGroups(X, Y)
ArrayList<List<Signals>> a
List<Signals> temp
query=sqlLiteExecute(THE_SQL_SNIPPET, x, y)
row=query.fetch() //fetch the first row to set the groupCounter
actualGroup=row.group
temp.add(row.signal)
for(row : query) //foreach row add the signal to the list
if(row.group!=actualGroup) //or reset the list if is a new group
a.add(actualGroup, temp)
actualGroup=row.group; temp= new List
temp.add(row.signal)
return a