Pass In "WHERE" parameters to PostgreSQL View? - sql

I have a rather complicated query on my PostgreSQL database spanning 4 tables via a series of nested subqueries. However, despite the slightly tricky looking appearance and setup, ultimately it will return two columns (from the same table, if that helps the situation) based on that matching of two external parameters (two strings need to match with fields in different tables). I'm fairly new to database design in PostgreSQL, so I know that this seemingly magical thing called Views exist, and that seems like it could help me here, but perhaps not.
Is there some way I can move my complex query inside a view and somehow just pass it the two values I need to match? That would greatly simplify my code on the front-end (by shifting the complexities to the database structure). I can create a view that wraps my static example query, and that works just fine, however that only works for one pair of string values. I need to be able to use it with a variety of different values.
Thus my question is: is it possible to pass parameters into an otherwise static View and have it become "dynamic"? Or perhaps a View is not the right way to approach it. If there's something else that would work better, I'm all ears!
*Edit: * As requested in comments, here's my query as it stands now:
SELECT param_label, param_graphics_label
FROM parameters
WHERE param_id IN
(SELECT param_id
FROM parameter_links
WHERE region_id =
(SELECT region_id
FROM regions
WHERE region_label = '%PARAMETER 1%' AND model_id =
(SELECT model_id FROM models WHERE model_label = '%PARAMETER 2%')
)
) AND active = 'TRUE'
ORDER BY param_graphics_label;
Parameters are set off by percent symbols above.

You could use a set returning function:
create or replace function label_params(parm1 text, parm2 text)
returns table (param_label text, param_graphics_label text)
as
$body$
select ...
WHERE region_label = $1
AND model_id = (SELECT model_id FROM models WHERE model_label = $2)
....
$body$
language sql;
Then you can do:
select *
from label_params('foo', 'bar')
Btw: are you sure you want:
AND model_id = (SELECT model_id FROM models WHERE model_label = $2)
if model_label is not unique (or the primary key) then this will throw an error eventually. You probably want:
AND model_id IN (SELECT model_id FROM models WHERE model_label = $2)

In addition to what #a_horse already cleared up, you could simplify your SQL statement with JOIN syntax instead of nested subqueries. Performance will be similar, but the syntax is much shorter and easier to manage.
CREATE OR REPLACE FUNCTION param_labels(_region_label text, _model_label text)
RETURNS TABLE (param_label text, param_graphics_label text)
LANGUAGE sql AS
$func$
SELECT p.param_label, p.param_graphics_label
FROM parameters p
JOIN parameter_links l USING (param_id)
JOIN regions r USING (region_id)
JOIN models m USING (model_id)
WHERE p.active
AND r.region_label = $1
AND m.model_label = $2
ORDER BY p.param_graphics_label;
$func$;
If model_label is not unique or something else in the query produces duplicate rows, you may want to make that SELECT DISTINCT p.param_graphics_label, p.param_label - with a matching ORDER BY clause for best performance. Or use GROUP BY.
Since Postgres 9.2 you can use the declared parameter names in place of $1 and $2 in SQL functions. (Has been possible for PL/pgSQL functions for a long time).
To avoid naming conflicts, I prefix parameter names with _ (those are visible most everywhere inside the function) and table-qualify column names in queries.
I simplified WHERE p.active = 'TRUE' to WHERE p.active, assuming the column active is type boolean.
USING in the JOIN condition only works if the column names are unambiguous across all tables to the left. Else use the more explicit syntax: ON l.param_id = p.param_id

In most cases the set-returning function is the way to go, but in the event that you want to both read and write to the set, a view may be more appropriate. And it is possible for a view to read a session parameter:
CREATE VIEW widget_sb AS
SELECT * FROM widget
WHERE column = cast(current_setting('mydomain.myparam') as int)
SET mydomain.myparam = 0
select * from widget_sb
[results]
SET mydomain.myparam = 1
select * from widget_sb
[distinct results]

I don't think a "dynamic" view as you stated is possible.
Why not write a stored procedure that takes 2 arguments instead?

I would rephrase the query as the following:
SELECT p.param_label, p.param_graphics_label
FROM parameters p
where exists (
select 1
from parameter_links pl
where pl.parameter_id = p.id
and exists (select 1 from regions r where r.region_id = pl.region_id
) and p.active = 'TRUE'
order by p.param_graphics_label;
Assuming that you have indexes on the various id columns, this query should be significantly faster than using the IN operator; the exists parameters here will use only the index values without even touching the data table except for getting the final data from the parameters table.

Related

Oracle Invalid Number in Join Clause

I am getting an Oracle Invalid Number error that doesn't make sense to me. I understand what this error means but it should not be happening in this case. Sorry for the long question, but please bear with me so I can explain this thoroughly.
I have a table which stores IDs to different sources, and some of the IDs can contain letters. Therefore, the column is a VARCHAR.
One of the sources has numeric IDs, and I want to join to that source:
SELECT *
FROM (
SELECT AGGPROJ_ID -- this column is a VARCHAR
FROM AGG_MATCHES -- this is the table storing the matches
WHERE AGGSRC = 'source_a'
) m
JOIN SOURCE_A a ON a.ID = TO_NUMBER(m.AGGPROJ_ID);
In most cases this works, but depending on random things such as what columns are in the select clause, if it uses a left join or an inner join, etc., I will start seeing the Invalid Number error.
I have verified multiple times that all entries in AGG_MATCHES where AGGSRC = 'source_a' do not contain non numeric characters in the AGGPROJ_ID column:
-- this returns no results
SELECT AGGPROJ_ID
FROM AGG_MATCHES
WHERE AGGSRC = 'source_a' AND REGEXP_LIKE(AGGPROJ_ID, '[^0-9]');
I know that Oracle basically rewrites the query internally for optimization. Going back to the first SQL example, my best guess is that depending on how the entire query is written, in some cases Oracle is trying to perform the JOIN before the sub query. In other words, it's trying to join the entire AGG_MATCHES tables to SOURCE_A instead of just the subset returned by the sub query. If so, there would be rows that contain non numeric values in the AGGPROJ_ID column.
Does anyone know for certain if this is what's causing the error? If it is the reason, is there anyway for me to force Oracle to execute the sub query part first so it's only trying to join a subset of the AGG_MATCHES table?
A little more background:
This is obviously a simplified example to illustrate the problem. The AGG_MATCHES table is used to store "matches" between different sources (i.e. projects). In other words, it's used to say that a project in sourceA is matched to a project in sourceB.
Instead of writing the same SQL over and over, I've created views for the sources we commonly use. The idea is to have a view with two columns, one for SourceA and one for SourceB. For this reason, I don't want to use TO_CHAR on the ID column of the source table, because devs would have to remember to do this every time they are doing a join, and I'm trying to remove code duplication. Also, since the ID in SOURCE_A is a number, I feel that any view storing SOURCE_A.ID should go ahead and convert it to a number.
You are right that Oracle is executing the statement in a different order than what you wrote, causing conversion errors.
The best ways to fix this problem, in order, are:
Change the data model to always store data as the correct type. Always store numbers as numbers, dates as dates, and strings as strings. (You already know this and said you can't change your data model, this is a warning for future readers.)
Convert numbers to strings with a TO_CHAR.
If you're on 12.2, convert strings to numbers using the DEFAULT return_value ON CONVERSION ERROR syntax, like this:
SELECT *
FROM (
SELECT AGGPROJ_ID -- this column is a VARCHAR
FROM AGG_MATCHES -- this is the table storing the matches
WHERE AGGSRC = 'source_a'
) m
JOIN SOURCE_A a ON a.ID = TO_NUMBER(m.AGGPROJ_ID default null on conversion error);
Add a ROWNUM to an inline view to prevent optimizer transformations that may re-write statements. ROWNUM is always evaluated at the end and it forces Oracle to run things in a certain order, even if the ROWNUM isn't used. (Officially hints are the way to do this, but getting hints right is too difficult.)
SELECT *
FROM (
SELECT AGGPROJ_ID -- this column is a VARCHAR
FROM AGG_MATCHES -- this is the table storing the matches
WHERE AGGSRC = 'source_a'
--Prevent optimizer transformations for type safety.
AND ROWNUM >= 1
) m
JOIN SOURCE_A a ON a.ID = TO_NUMBER(m.AGGPROJ_ID);
I think the simplest solution uses case, which has more guarantees on the order of evaluation:
SELECT a.*
FROM AGG_MATCHES m JOIN
SOURCE_A a
ON a.ID = (CASE WHEN m.AGGSRC = 'source_a' THEN TO_NUMBER(m.AGGPROJ_ID) END);
Or, better yet, convert to strings:
SELECT a.*
FROM AGG_MATCHES m JOIN
SOURCE_A a
ON TO_CHAR(a.ID) = m.AGGPROJ_ID AND
m.AGGSRC = 'source_a' ;
That said, the best advice is to fix the data model.
Possibly the best solution in your case is simply a view or a generate column:
create view v_agg_matches_a as
select . . .,
(case when regexp_like(AGGPROJ_ID, '^[0-9]+$')
then to_number(AGGPROJ_ID)
end) as AGGPROJ_ID
from agg_matches am
where m.AGGSRC = 'source_a';
The case may not be necessary if you use a view, but it is safer.
Then use the view in subsequent queries.

How to join var array values

SELECT
person.bu_id,
key.key_val,
obj_person_add.person_contact_freq,
obj_person_add.person_contact_best,
OBJ_PERSON_ADD.PERSON_CONTACT_INST,
OBJ_PERSON_ADD.PERSON_CONTACT_DATE,
OBJ_PERSON_ADD.PERSON_BROCHURES,
OBJ_PERSON_ADD.PERSON_NEWSLETTER
FROM obj_person person
INNER JOIN obj_person_add
ON person.obj_id = obj_person_add.obj_id
LEFT JOIN obj_rel_key key
ON key.obj_id = person.obj_id
WHERE person.bu_id in (6,7)
AND person.close_date IS NULL
AND key.obj_key_id = 806
AND (
obj_person_add.person_contact_freq IS NOT NULL
OR obj_person_add.person_contact_best IS NOT NULL
OR OBJ_PERSON_ADD.PERSON_CONTACT_INST IS NOT NULL
OR OBJ_PERSON_ADD.PERSON_CONTACT_DATE IS NOT NULL
OR OBJ_PERSON_ADD.PERSON_BROCHURES IS NOT NULL
OR OBJ_PERSON_ADD.PERSON_NEWSLETTER IS NOT NULL
);
I have built this query, however person_brochures and person_newsletters are stored in a var array. This means that they return (6172,6544...) for example. I want to do a left join to match the ID's in the var array to a name in a different table.
Do I have to loop the var array and somehow match the ID's then?
You might try something called collection unnesting:
SELECT
person.bu_id,
...
OBJ_PERSON_ADD.PERSON_BROCHURES,
OBJ_PERSON_ADD.PERSON_NEWSLETTER
FROM obj_person person
INNER JOIN obj_person_add
ON person.obj_id = obj_person_add.obj_id
...,
TABLE(OBJ_PERSON_ADD.PERSON_BROCHURES) x
INNER JOIN your_other_table
ON x.column_value = your_other_table.id
...
column_value is the name of Oracle pseudocolumn that is assigned to the VARRAY elements.
I am not 100% clear, I am guessing that you are writing this code in C# .Net.
You need to decide where the problem is going to be solved in , are you going to solve it in your App tier or persistence tier ?
I am assuming that you want to solve it in the persistence tier , you have couple of options
1. Generate a prepared statement and execute the statement ..instead of join you will have to rely on IN operator ..etc
Use a stored procedure , pass the two arrays as comma separated string to a stored procedure
you can look at this example if you need to solve it in Oracle function ..so you can do a join as you originally thought.
Splitting string into multiple rows in Oracle
use oracle object relational features and map the data structure appropriately to a structure and use oracle MULTISET like collections to solve the query in a stored procedure ..etc

Alternate solution for the query - Used INTERSECT function in oracle plsql

I am working on the query. I have two tables one is detail table where not grouping happen and its like including all the values and other table is line table which has important column grouped together from detail table.
I want to show all the column from line table and some column from detail table.
I am using below query to fetch my records
SELECT ab.*,
cd.phone_number,
cd.id
FROM xxx_line ab,
xxx_detail cd
WHERE cd.reference_number = ab.reference_number
AND cd.org_id = ab.org_id
AND cd.request_id = ab.request_id
AND ab.request_id = 13414224
INTERSECT
SELECT ab.*,
cd.phone_number,
cd.id
FROM xxx_line ab,
xxx_detail cd
WHERE cd.reference_number = ab.reference_number
AND cd.org_id = ab.org_id
AND cd.request_id = ab.request_id
AND ab.request_id = 13414224
The query is working fine...
But I want to know is there any other way for I can achieve the same result by not even using Intersect.
I purpose is to find out all possible way to get the same output.
The INTERSECT operator returns the unique set of rows returned by each query. The code can be re-written with a DISTINCT operator to make the meaning clearer:
SELECT DISTINCT
xxx_line.*,
xxx_detail.phone_number,
xxx_detail.id
FROM xxx_line
JOIN xxx_detail
ON xxx_line.reference_number = xxx_detail.reference_number
AND xxx_line.org_id = xxx_detail.org_id
AND xxx_line.request_id = xxx_detail.request_id
WHERE xxx_line.request_id = 13414224
I also replaced the old-fashioned join syntax with the newer ANSI join syntax (which makes relationships clearer by forcing the join tables and conditions to be listed close to each other) and removed the meaningless table aliases (because code complexity is more directly related to the number of variables than the number of characters).

In an EXISTS can my JOIN ON use a value from the original select

I have an order system. Users with can be attached to different orders as a type of different user. They can download documents associated with an order. Documents are only given to certain types of users on the order. I'm having trouble writing the query to check a user's permission to view a document and select the info about the document.
I have the following tables and (applicable) fields:
Docs: DocNo, FileNo
DocAccess: DocNo, UserTypeWithAccess
FileUsers: FileNo, UserType, UserNo
I have the following query:
SELECT Docs.*
FROM Docs
WHERE DocNo = 1000
AND EXISTS (
SELECT * FROM DocAccess
LEFT JOIN FileUsers
ON FileUsers.UserType = DocAccess.UserTypeWithAccess
AND FileUsers.FileNo = Docs.FileNo /* Errors here */
WHERE DocAccess.UserNo = 2000 )
The trouble is that in the Exists Select, it does not recognize Docs (at Docs.FileNo) as a valid table. If I move the second on argument to the where clause it works, but I would rather limit the initial join rather than filter them out after the fact.
I can get around this a couple ways, but this seems like it would be best. Anything I'm missing here? Or is it simply not allowed?
I think this is a limitation of your database engine. In most databases, docs would be in scope for the entire subquery -- including both the where and in clauses.
However, you do not need to worry about where you put the particular clause. SQL is a descriptive language, not a procedural language. The purpose of SQL is to describe the output. The SQL engine, parser, and compiler should be choosing the most optimal execution path. Not always true. But, move the condition to the where clause and don't worry about it.
I am not clear why do you need to join with FileUsers at all in your subquery?
What is the purpose and idea of the query (in plain English)?
In any case, if you do need to join with FileUsers then I suggest to use the inner join and move second filter to the WHERE condition. I don't think you can use it in JOIN condition in subquery - at least I've never seen it used this way before. I believe you can only correlate through WHERE clause.
You have to use aliases to get this working:
SELECT
doc.*
FROM
Docs doc
WHERE
doc.DocNo = 1000
AND EXISTS (
SELECT
*
FROM
DocAccess acc
LEFT OUTER JOIN
FileUsers usr
ON
usr.UserType = acc.UserTypeWithAccess
AND usr.FileNo = doc.FileNo
WHERE
acc.UserNo = 2000
)
This also makes it more clear which table each field belongs to (think about using the same table twice or more in the same query with different aliases).
If you would only like to limit the output to one row you can use TOP 1:
SELECT TOP 1
doc.*
FROM
Docs doc
INNER JOIN
FileUsers usr
ON
usr.FileNo = doc.FileNo
INNER JOIN
DocAccess acc
ON
acc.UserTypeWithAccess = usr.UserType
WHERE
doc.DocNo = 1000
AND acc.UserNo = 2000
Of course the second query works a bit different than the first one (both JOINS are INNER). Depeding on your data model you might even leave the TOP 1 out of that query.

sql join question

I have the following tables
nid timestamp title
82 1245157883 Home
61 1245100302 Minutes
132 1245097268 Sample Form
95 1245096985 Goals & Objectives
99 1245096952 Members
AND
pid src dst language
70 node/82 department/34-section-2
45 node/61/feed department/22-section-2/feed
26 node/15 department/department1/15-department1
303 node/101 department/101-section-4
These are fragments of the tables, and is missing the rest of the data (they are both quite large), but I am trying to join the dst column from the second table into the first one. They should match up on their "nid", but the second table has node/[nid] which makes this more complicated. I also want to ignore the ones that end in "feed" since they are not needed for what I am doing.
Much thanks
EDIT: I feel bad for not mentioning this, but the first table is an sql result from
select nid, MAX(timestamp) as timestamp, title from node_revisions group by nid ORDER BY timestamp DESC LIMIT 0,5
The second table has the name "url_alias"
try
select * from table1 inner join table2 on src=concat('node/',nid)
Edit
edited to reflect change in OP
select `nid`, MAX(`timestamp`) as `timestamp`, `title` from `node_revisions` inner join `url_alias` on `src`=concat('node/',`nid`) group by `nid` ORDER BY `timestamp` DESC LIMIT 0,5
I don't know what database you are using. However, I suggest you write a parsing function that returns the nid from that column. Then, you can have this kind of query (assuming GET_NID is the function you defined):
SELECT * from T1, T2
WHERE T1.nid = GET_NID( T2.node)
You have a few options.
write a function that converts src to an nid and join on t1.nid = f(t2.src) -- you didn't say what DBMS you use, but most have a way to do that. It will be slow, but that depends on how big the tables are.
Similar to that, make a view that has a computed field using that function -- same speed, but might be easier to understand.
Create a new nid field in t2 and use the function to populate it. Make insert and update triggers to keep it up to date, then join on that. This is better if you query this frequently.
Convert t2 so that it has a nid field and compute the src from that and another field that is a template that the nid needs to be inserted into.
I'd pull the node id in the second table into a separate column. Otherwise any attempt to join the two tables will result in a table scan with some processing on the src field (I assume you meant the src field and not the dst field) and performance will be problematic.
SELECT *
FROM (SELECT *, 'node/' + nid AS src FROM table1) t1
INNER JOIN table2 t2
ON t1.src = t2.src
You haven't specified with DBMS are you using. Most engines support the SQL-99 standard SIMILAR TO clause which is using regular expression for matching. Some engines also implement this, but use some other keywords instead of SIMILAR TO.
FirebirdSQL:
http://wiki.firebirdsql.org/wiki/index.php?page=SIMILAR+TO
PostgreSQL:
http://www.network-theory.co.uk/docs/postgresql/vol1/SIMILARTORegularExpressions.html
MySQL:
http://dev.mysql.com/doc/refman/5.0/en/regexp.html
Depending on the scenario you want to this for (if for example you are regularly going to be performing this JOIN and your 2nd table is rather large) you may want to look into a Materialized View.
Write a function that performs all the logic to extract the nid into a separate column. Aside from initial m-view creation, the function will only need to run when the basetable changes (insert, update, delete) compared to running the function against every row each time you query.
This allows a fairly simple join to the materialized view with standard benefits of tables such as Indexing.
NB: looks like I was beaten to it while writing :)