PostGIS 2.3 Split a line by points - sql

I asked this question on gis.stackexchange ( but since my actual problem seems to be more a DB problem than GIS I am trying my luck here). Here is the question on gis.stackexchange : https://gis.stackexchange.com/questions/256535/postgis-2-3-splitting-multiline-by-points
I have a trigger in which I a looping when inserting a new line to INSERT the set of splitted lines in my table, but for some reason I do not get the wanted result since in the example I only get two lines out of three. What a I doing wrong ?
Here comes the code of the trigger function :
CREATE OR REPLACE FUNCTION public.split_cable()
RETURNS trigger AS
$BODY$
DECLARE compte integer;
DECLARE i integer := 2;
BEGIN
compte = (SELECT count(*) FROM boite WHERE st_intersects(boite.geom, new.geom));
WHILE i < compte LOOP
WITH brs AS (SELECT row_number() over(), boite.geom FROM boite, cable2
WHERE st_intersects(boite.geom, new.geom)
-- here the ORDER BY serve to get the "boite" objects in a specific order
ORDER BY st_linelocatepoint(st_linemerge(new.geom),boite.geom)),
brs2 AS (SELECT st_union(geom) AS geom FROM brs),
cables AS (SELECT (st_dump(st_split(new.geom, brs2.geom))).geom FROM brs2)
INSERT INTO cable2 (geom) VALUES (
SELECT st_multi(cables.geom) FROM cables WHERE st_startpoint(geom) = (SELECT geom FROM brs WHERE brs.row_number = i));
i = i + 1;
END LOOP;
new.geom = (WITH brs AS (SELECT row_number() over(), boite.geom FROM boite, cable2
WHERE st_intersects(boite.geom, new.geom)
ORDER BY st_linelocatepoint(st_linemerge(new.geom),boite.geom)),
brs2 AS (SELECT st_union(geom) as geom from brs),
cables AS (SELECT (st_dump(st_split(new.geom, brs2.geom))).geom FROM brs2)
SELECT st_multi(cables.geom) FROM cables WHERE st_startpoint(geom) = (SELECT geom FROM brs WHERE brs.row_number = 1));
RETURN new;
END
$BODY$
LANGUAGE plpgsql;

This is a relatively complex query and has a lot of moving parts.
my recommendation for debugging the query involves multiple ideas:
Consider splitting the function into smaller functions, that are easier to test, and then compose the function from a set of parts you know for sure work as you need them to.
export a set of intermediate results to an intermediate table, you you can visualise the intermediate result-sets easily using a graphical tool and can better assess where the data went wrong.
is is possible that the combination of ST_ functions you are using don't create the geometries you think they create, one way to rule this out is by visualising the results of geographical function combinations, like st_dump(st_split(...))) or st_dump(st_split(...)) for example.
perhaps this check: st_startpoint(geom) = (SELECT geom FROM brs WHERE brs.row_number = i)) could be made by checking "points near" and not "exact point", maybe the points are very near, as in centimeters near, making them essentially "the same point", but not actually be the exact point. this is just an assumption though.
Consider sharing more data with StackOverflow! like a small dataset or example so we can actually run the code! :)

Related

Query N closest database entries using active record query interface

In my Rails project I have a table called "items", where each item has a latitude and longitude property.
When I retrieve the items for my client, I fetch the current user's longitude and latitude. Using these two sets of coordinates, I want to use a haversine function and fetch 100 results with the smallest haversine distance.
The SQL should look something like this:
SELECT
*,
haversine_distance(user_lat, user_long, item_lat, item_long) as distance
FROM
db.items
WHERE
item_lat < {user_lat} +1
AND item_lat > {user_lat}+1
AND ..
AND ..
ORDER BY distance DESC
LIMIT 100
but this would necessitate using a custom function in the query, which I'm not sure how to do.
Also, regarding lines 7 and 8 of the query, I'm attempting to a where clause to quickly filter the set of results within some reasonable distance so I don't have to apply the haversine to the entire table.
Right now, I have the function defined in a singleton called DistanceService, but I assume I cannot use Ruby functions in the query.
When I tried to add the function haversine_distance to the controller and call it like:
#items = Item.select("items.*, haversine_distance(geo_long, geo_lat,
#{#current_user.geo_long}, #{#current_user.geo_lat}) AS distance")
.where(status: 'available', ....)
I was met with
FUNCTION ****_api_development.haversine_distance does not exist. This leads me to think the function should be somehow defined in SQL first via some migration, but I'm not very familiar with SQL and don't know how I would do that with a migration.
I'm also wondering if there's a solution that would support pagination, as it will hopefully eventually become a requirement for my project.
Thanks.
In Rails you define database functions through migrations:
class AddHaversineDistanceFunction < ActiveRecord::Migration
def up
execute <<~SQL
DELIMITER $$
DROP FUNCTION IF EXISTS haversine_distance$$
CREATE FUNCTION haversine_distance(
lat1 FLOAT, lon1 FLOAT,
lat2 FLOAT, lon2 FLOAT
) RETURNS FLOAT
NO SQL DETERMINISTIC
COMMENT 'Returns the distance in km
between two known points of latitude and longitude'
BEGIN
RETURN DEGREES(ACOS(
COS(RADIANS(lat1)) *
COS(RADIANS(lat2)) *
COS(RADIANS(lon2) - RADIANS(lon1)) +
SIN(RADIANS(lat1)) * SIN(RADIANS(lat2))
)) * 111.045;
END$$
DELIMITER;
SQL
end
def down
execute "DROP FUNCTION IF EXISTS haversine_distance"
end
end
The actual function here is adapted from Plum Island Media.
But you might want to check out the geocoder gem which provides this functionality and uses a better formula.

I can't figure out what v_out and v_in mean in a SQL statement

know some basic SQL enough to fix some issues or at least understand how it works, but I got confused with this. I figured out that they're some kind of commands since they're not tables. But I can't find anywhere here or first three pages of Google what exactly does v_in and v_out do.
The most confusing part is:
--some code above
inv_docs.datdoc AS day,
inv_docs.nrdoc AS doc_num,
v_out.article_id AS article_id,
v_out.qty_out AS sent,
--some code bellow
Where did v_out come from? To what is it related? Also bellow where I get to FROM part:
FROM inv_docs_out v_out
JOIN inv_docs_in v_in ON v_out.lotid = v_in.idlot
Now I have v_out from nowhere, and it's not even connected to inv_docs via a dot (eg. inv_docs.v_in).
Those are table aliases, for example if you do
SELECT * FROM table_with_long_name t
You can use t as an alias. For example
SELECT t.date FROM table_with_long_name t WHERE t.name = 'A Name'
So in your case, what the following query does is letting you use v_out instead of inv_docs_out and v_in instead of inv_docs_in.
FROM inv_docs_out v_out
JOIN inv_docs_in v_in ON v_out.lotid = v_in.idl

PostgreSQL Mathematical Function

I have a table aps_sections with many integer fields (such as bare_width and worn_width). I also have multiple look up tables (such as aps_bare_width and aps_worn_width) which contain an ID column and a WEIGHTING column. The ID is recorded in the above columns of the aps_sections table. I need to sum the WEIGHTINGs of the columns in the aps_sections table (whereby the WEIGHTING value comes from the look up tables). I have successfully managed this using the below SELECT statement.
SELECT aps_sections.ogc_fid,
( aps_bare_width.weighting
+ aps_worn_width.weighting
+ aps_gradient.weighting
+ aps_braiding.weighting
+ aps_pigeon.weighting
+ aps_depth.weighting
+ aps_standing_water.weighting
+ aps_running_water.weighting
+ aps_roughness.weighting
+ aps_surface.weighting
+ aps_dynamic.weighting
+ aps_ex_cond.weighting
+ aps_promotion.weighting
+ aps_level_of_use.weighting) AS calc
FROM row_access.aps_sections,
row_access.aps_bare_width,
row_access.aps_worn_width,
row_access.aps_gradient,
row_access.aps_braiding,
row_access.aps_pigeon,
row_access.aps_depth,
row_access.aps_standing_water,
row_access.aps_running_water,
row_access.aps_roughness,
row_access.aps_surface,
row_access.aps_dynamic,
row_access.aps_ex_cond,
row_access.aps_promotion,
row_access.aps_level_of_use
WHERE aps_bare_width.fid = aps_sections.bare_width
AND aps_worn_width.fid = aps_sections.worn_width
AND aps_gradient.fid = aps_sections.gradient
AND aps_braiding.fid = aps_sections.braiding
AND aps_pigeon.fid = aps_sections.pigeon
AND aps_depth.fid = aps_sections.depth
AND aps_standing_water.fid = aps_sections.standing_water
AND aps_running_water.fid = aps_sections.running_water
AND aps_roughness.fid = aps_sections.roughness
AND aps_surface.fid = aps_sections.surface
AND aps_dynamic.fid = aps_sections.dynamic
AND aps_ex_cond.fid = aps_sections.ex_cond
AND aps_promotion.fid = aps_sections.promotion
AND aps_level_of_use.fid = aps_sections.level_of_use
What I now need to do is create a function that adds the calculated result to the physical_sn_priority column of the aps_sections table. My understanding so far is that my function should look similar to:
CREATE OR REPLACE FUNCTION row_access.aps_weightings()
RETURNS trigger AS
$BODY$
BEGIN
NEW.physical_sn_priority := ;
RETURN NEW;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
ALTER FUNCTION public.update_km()
OWNER TO postgres;
But I don't know what to put after NEW.physical_sn_priority :=. I am a beginner to SQL and to PostgreSQL so I would appreciate any guidance!
While Erwin is (as always) correct that a version would be helpful, I think your answer will be simplest with the SELECT ... INTO construction for PL/pgSQL. (Not the same as SELECT INTO that works like INSERT or CREATE TABLE.)
SELECT ( aps_bare_width.weighting
+ /* obvious deletia */
+ aps_level_of_use.weighting)
INTO NEW.physical_sn_priority
FROM row_access.aps_bare_width,
/* snip */,
row_access.aps_level_of_use
WHERE aps_bare_width.fid = NEW.bare_width
AND /* snip */
aps_level_of_use.fid = NEW.level_of_use;
RETURN NEW;
According to the documentation, the INTO can appear in several other places in the line; I find this simple to understand.
[EDIT]
While this works, on reflection, I think the schema should be revised.
CREATE TYPE weighted_item_t AS ENUM ('bare_width', /* ... */, 'level_of_use');
CREATE TABLE current_weights(item_type weighted_item_t, fid int, current_weight float);
/* key and checks omitted */
/* note, if item_type can be deduced from fid, we don't even need the enum */
CREATE TABLE sections_items(section_id int /* FK into aps_sections */,
item_type weighted_item_t, fid int);
Now the queries are going to collapse into simple sums. You need to insert records into section_items before aps_sections, which can be done with deferred constraints in a transaction with or without a stored procedure, depending on how you acquire the data and how much control you have over its format. If (and this is not clear, because it won't change on updates) you want the denormalized total, you can get it with
SELECT SUM(current_weight) INTO NEW.physical_sn_priority
FROM section_items NATURAL JOIN current_weights
WHERE NEW.section_id=section_items.section_id;
This will work out much better if additional weighted characteristics are added at some future date.
Simplified test case
You should present your question with less noise. This shorter query does the job just fine:
SELECT aps_sections.ogc_fid,
( aps_bare_width.weighting
+ aps_worn_width.weighting
+ aps_gradient.weighting) AS calc
FROM row_access.aps_sections,
row_access.aps_bare_width,
row_access.aps_worn_width,
row_access.aps_gradient,
WHERE aps_bare_width.fid = aps_sections.bare_width
AND aps_worn_width.fid = aps_sections.worn_width
AND aps_gradient.fid = aps_sections.gradient;
Answer
As #Andrew already advised, the golden way would be to simplify your schema.
If, for some reason, this is not possible, here is an alternative to simplify the addition of many columns (which may or may not be NULL), create this tiny, powerful function:
CREATE OR REPLACE FUNCTION f_sum(ANYARRAY)
RETURNS numeric LANGUAGE sql AS
'SELECT sum(i)::numeric FROM unnest($1) i';
Call:
SELECT f_sum('{2,NULL,7}'::int[])
It takes a polymorphic array type and returns numeric. Works for any number type that can be summed by sum(). Cast the result if needed.
Simplifies the syntax for summing lots of columns.
NULL values won't break your calculation because the aggregate function sum() ignores those.
In a trigger function this could be used like this:
NEW.physical_sn_priority := f_sum(ARRAY [
COALESCE(physical_sn_priority, 0) -- add to original value
,(SELECT weighting FROM aps_bare_width x WHERE x.fid = NEW.bare_width)
,(SELECT weighting FROM aps_worn_width x WHERE x.fid = NEW.worn_width)
,(SELECT weighting FROM aps_gradient x WHERE x.fid = NEW.gradient)
...
])
Since all your joined tables are only good for looking up a single field, and completely independent from each other, you can just as well use individual subqueries. I also went this way, because we do not know whether any of the subqueries might return NULL, in which case your original query or Andrew's version would result in no row / no assignment.
Assignment to NEW really only makes sense in a BEFORE trigger on the table aps_sections. This code works BEFORE INSERT (where the row cannot be found in the table yet!) as well as BEFORE UPDATE. You want to use the current values from NEW, not the old row version in the table.

Delphi query parameter usage when all values is also an option

I have a tquery (going thru BDE or BDE emulating component) that has been used to select either a single record or all records.
Traditionally this has been done as such:
select * from clients where (clientid = :clientid or :clientid = -1)
And then they would put a -1 in the field when they wanted the query to return all values. Going through this code though, I have discovered that when they have done this the query does not use proper indexing for the table and only does a natural read.
Is there a best practices method for achieving this? Perhaps a way to tell a parameter to return all values, or must the script be modified to remove the where clause entirely when all values are desired?
Edit: This is Delphi 7, by the way (And going against Firebird 1.5 sorry for leaving that out)
As you use deprecated BDE, that may be one more reason to migrate from BDE to 3d party solutions. AnyDAC (UniDAC, probably others too. Most are commercial libraries) has macros, which allow to dynamically change a SQL command text, depending on the macro values. So, your query may be written:
ADQuery1.SQL.Text := 'select * from clients {IF &clientid} where clientid = &clientid {FI}';
if clientid >= 0 then
// to get a single record
ADQuery1.Macros[0].AsInteger := clientid
else
// to get all records
ADQuery1.Macros[0].Clear;
ADQuery1.Open;
For the queries with "optional" parameters I always use ISNULL (MSSQL, or NVL Oracle), ie.
SELECT * FROM clients WHERE ISNULL(:clientid, clientid) = clientid
Setting the parameter to NULL then selects all records.
You also have to take care of NULL values in the table fields because NULL <> NULL. This you can overcome with a slight modification:
SELECT * FROM clients WHERE COALESCE(:clientid, clientid, -1) = ISNULL(clientid, -1)
I would use this:
SELECT * FROM CLIENTS WHERE clientid = :clientid or :clientid IS NULL
Using two queries is best:
if (clientid <> -1) then
begin
DBComp.SQL.Text := 'select * from clients where (clientid = :clientid)';
DBComp.ParamByName('clientid').Value := clientid;
end else
begin
DBComp.SQL.Text := 'select * from clients';
end;
DBComp.Open;
...
Alternatively:
DBComp.SQL.BeginUpdate;
try
DBComp.SQL.Clear;
DBComp.SQL.Add('select * from clients');
if (clientid <> -1) then
DBComp.SQL.Add('where (clientid = :clientid)');
finally
DBComp.SQL.EndUpdate;
end;
if (clientid <> -1) then
DBComp.ParamByName('clientid').Value := clientid;
DBComp.Open;
...
Remy's answer may be re-formulated as single query.
It may be better, if you gonna prepare it once and then re-open multiple times.
select * from clients where (clientid = :clientid)
and (:clientid is not null)
UNION ALL
select * from clients where (:clientid is null)
This just aggregates two distinct queries (with same results vector) together. And condition just turns one of those off.
Using would be like that:
DBComp.Prepare.
...
DBComp.Close;
DBComp.ParamByName('clientid').Value := clientid;
DBComp.Open;
...
DBComp.Close;
DBComp.ParamByName('clientid').Clear;
DBComp.Open;
However this query would rely on SQL Server optimizer capability to extract query invariant (:clientid is [not] null) and enable/disable query completely. But well, your original query depends upon that too.
Why still use obsolete FB 1.5 ? Won't FB 2.5.2 work better there ?
I think your original query is formulated poorly.
select * from clients where (:clientid = -1) or ((clientid = :clientid) and (:clientid <> -1))
would probably be easier on SQL Server optimizer. Yet i think FB could do better job there. Try to download later FB, and run your query in it, using IDEs like IBExpert or FlameRobin. Re-arranging parenthesis and changing -1 to NULL are obvious ideas to try.
Using BDE is fragile now. It is not very fast, limiting in datatypes and connectivity (no FB/IB Events for example). And would have all sorts of compatibility problems with Vista/Win7 and Win64. If FB/IB is your server of choice, consider switching to some modern component set:
(FLOSS) Universal Interbase by http://uib.sf.net (RIP all Delphi pages of http://Progdigy.com )
(FLOSS) ZeosLib DBO by http://zeos.firmos.at/
(propr) FIB+ by http://DevRace.com
(propr) IB Objects by http://IBobjects.com
(propr) AnyDAC by http://da-soft.com - sold out to Embarcadero, not-avail for D7
(propr) IB-DAC/UniDAC http://DevArt.com
Also it would be good thing to show the table and indices definition and selectivity of those indices.

PL/SQL - use same pipelined function twice in the same query

I'm trying to use a pipelined function to save on time and reduce redundancy in my queries.
The function in question returns data from a reference table based on some input. Records in the main data table I am selecting from have multiple columns that all refer to the reference table. The problem I run into is that when I try to use the pipelined function more than once in the query, I get a "cursor already open" error.
For example:
select xmlelement("genInf", xmlelement("ID", vt.ID),
xmlelement("vID", vt.V_ID),
xmlelement("vNum", vt.V_NUM),
xmlelement("terrDataCode", TERR_CODE.column_value), --data is based on reference table
xmlelement("ABValCode", AB_VAL_CD.column_value), --data is based on reference table
...
from V_TAB vt, table(UTIL.fn_getOvrdValXML(vt.terr_cd_id)) TERR_CODE,
table(UTIL.fn_getOvrdValXML(vt.ab_val_id)) AB_VAL_CD
where vt.ID = in_vID;
This worked fine until I added the second reference to my pipeline function (fn_getOvrdValXML), and I now get the "cursor already open" error.
The pipelined function is very simple:
type t_XMLTab is table of XMLType; --this type is in the spec
....
function fn_getOvrdValXML(in_ID in ovrd.id%type) return t_XMLTab
pipelined is
begin
for r in C_OvrdVal(in_ID) loop
pipe row(r.XMLChunk);
end loop;
return;
end;
The cursor is similarly simple:
cursor C_OvrdVal(in_ID in ovrd.id%type) is
select xmlforest(ID as "valueID", S_VAL as "sValue", U_VAL as "uplValue",
O_VAL as "oValue", O_IND as "oIndicator", F_VAL as "finalValue",
O_RSN as "reason") AS XMLChunk
from ovrd_val xov;
where xov.id = in_ID;
Is there a way to work around this, or should I try to tackle this problem (the problem of having to reference ovrd_val and output an xmlforest in the same way many many many many times) differently?
I admit I'm new to pipelined functions so I'm not 100% sure this is an appropriate use, but it made sense at the time and I'm open to other ideas ;)
If you're using pipeline functions, then you're on 9i minimum which means you can use the WITH clause:
WITH ovrdValXML AS (
select xov.id,
xmlforest(ID as "valueID", S_VAL as "sValue", U_VAL as "uplValue",
O_VAL as "oValue", O_IND as "oIndicator", F_VAL as "finalValue",
O_RSN as "reason") AS XMLChunk
from ovrd_val xov)
SELECT xmlelement("genInf", xmlelement("ID", vt.ID),
xmlelement("vID", vt.V_ID),
xmlelement("vNum", vt.V_NUM),
xmlelement("terrDataCode", TERR_CODE.column_value), --data is based on reference table
xmlelement("ABValCode", AB_VAL_CD.column_value), --data is based on reference table
...
FROM V_TAB vt
JOIN ovrdValXML terr_code ON terr_code = vt.?
AND terr_code.id = vt.terr_cd_id
JOIN ovrdValXML ab_val_cd ON ab_val_cd = vt.?
AND ab_val_cd.id = vt.ab_val_cd
WHERE vt.id = IN_VID;
Untested, and it's not clear what you're joining too - hence the ? on the join criteria.
Have you tried actually closing your cursor inside that pipelined function before piping row?
OPEN C_OvrdVal(in_ID);
FETCH c_OrdVal INTO my_chunk_variable;
CLOSE C_OrdVal;
PIPE ROW my_chunk_variable;
RETURN;