oracle multiple criteria dynamic sql - sql

Question

I have a multiple criteria search function for user input/select different criteria to find results, and every criteria is optional, so the field value could be null; The PL/SQL backend processes each criteria value to construct a dynamic SQL.
Currently, I use the below way to process, but it is hard for debugging and maintaining.
jo := json_object_t(p_payload);
v_country := jo.get_String('IAINST_NATN_CODE');
v_region := jo.get_String('IAINST_REGN_CODE');
v_rank_code := jo.get_String('RANK_CODE');
v_year := jo.get_String('RANK_YEAR');
v_sql := 'select * from IAVW_INST_JSON_TABLE i where
((:1 is null) or (i.IAINST_NATN_CODE = :1))
and ((:2 is null) or (i.IAINST_REGN_CODE = :2))
and ((:3 is null) or (i.RANK_CODE = :3))
and ((:4 is null) or (i.RANK_YEAR = :4))';
OPEN c FOR v_sql
USING v_country, v_country, --1
v_region, v_region, --2
v_rank_code, v_rank_code, --3
v_year, v_year; --4
RETURN c;
Any good advice to improve?

Answer 1

I would only change the structure of the clauses to be like :
AND i.IAINST_REGN_CODE = NVL(:2, i.IAINST_REGN_CODE)
This way you will avoid OR and still won't interfer with indexing if there is any, but apart from that your code looks fine (and fine even without my suggestion either).

Answer 2

After searching the related post. Here is the summary:
For my scenario, my table owns around 5K rows.
So
WHERE NVL(mycolumn,'NULL') = NVL(searchvalue,'NULL')
could simplify my dynamic SQL.
But if the table owns massive data, the above approach is not efficient (time cost to run the column conversion for NVL), please use the below query:
where ((MYCOLUMN=SEARCHVALUE) OR (MYCOLUMN is NULL and SEARCHVALUE is NULL))
Details see this post: Determine Oracle null == null
https://asktom.oracle.com/pls/apex/f?p=100:11:0::::P11_QUESTION_ID:7806711400346248708

Answer 3

For parameters referencing non-nullable columns you can use
and t.somecol = nvl(:b1,t.somecol)
For this the parser/optimiser will typically generate an execution plan with a union-all and a filter such that the most efficient approach will be used depending on whether :b1 is null or not (depending on database version, indexing, stats etc).
select * from bigtable t where t.product_type = nvl(:b1,t.product_type)
---------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost | Time |
---------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 6000000 | 486000000 | 5679 | 00:00:01 |
| 1 | PX COORDINATOR | | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ10000 | 6000000 | 486000000 | 5679 | 00:00:01 |
| 3 | VIEW | VW_ORE_1B35BA0F | 6000000 | 486000000 | 5679 | 00:00:01 |
| 4 | UNION-ALL | | | | | |
| * 5 | FILTER | | | | | |
| 6 | PX BLOCK ITERATOR | | 1200000 | 145200000 | 2840 | 00:00:01 |
| * 7 | TABLE ACCESS FULL | BIGTABLE | 1200000 | 145200000 | 2840 | 00:00:01 |
| * 8 | FILTER | | | | | |
| 9 | PX BLOCK ITERATOR | | 4800000 | 580800000 | 2840 | 00:00:01 |
| 10 | TABLE ACCESS FULL | BIGTABLE | 4800000 | 580800000 | 2840 | 00:00:01 |
---------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
------------------------------------------
* 5 - filter(:B1 IS NOT NULL)
* 7 - filter("T"."PRODUCT_TYPE"=:B1)
* 8 - filter(:B1 IS NULL)
However, it obviously can't keep extending this by generating union-alls for every possible combination of an arbitrarily large number of bind variables.
select * from bigtable t
where t.product_type = nvl(:b1,t.product_type)
and t.in_stock = nvl(:b2,t.in_stock)
and t.discounted = nvl(:b3,t.discounted)
---------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost | Time |
---------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 594 | 48114 | 5699 | 00:00:01 |
| 1 | PX COORDINATOR | | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ10000 | 594 | 48114 | 5699 | 00:00:01 |
| 3 | VIEW | VW_ORE_1B35BA0F | 594 | 48114 | 5699 | 00:00:01 |
| 4 | UNION-ALL | | | | | |
| * 5 | FILTER | | | | | |
| 6 | PX BLOCK ITERATOR | | 119 | 14399 | 2844 | 00:00:01 |
| * 7 | TABLE ACCESS FULL | BIGTABLE | 119 | 14399 | 2844 | 00:00:01 |
| * 8 | FILTER | | | | | |
| 9 | PX BLOCK ITERATOR | | 475 | 57475 | 2854 | 00:00:01 |
| * 10 | TABLE ACCESS FULL | BIGTABLE | 475 | 57475 | 2854 | 00:00:01 |
---------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
------------------------------------------
* 5 - filter(:B1 IS NOT NULL)
* 7 - filter("T"."PRODUCT_TYPE"=:B1 AND "T"."IN_STOCK"=NVL(:B2,"T"."IN_STOCK") AND "T"."DISCOUNTED"=NVL(:B3,"T"."DISCOUNTED") AND (NVL(:B2,"T"."IN_STOCK")='Y' OR NVL(:B2,"T"."IN_STOCK")='N') AND
(NVL(:B3,"T"."DISCOUNTED")='Y' OR NVL(:B3,"T"."DISCOUNTED")='N'))
* 8 - filter(:B1 IS NULL)
* 10 - filter("T"."IN_STOCK"=NVL(:B2,"T"."IN_STOCK") AND "T"."DISCOUNTED"=NVL(:B3,"T"."DISCOUNTED") AND (NVL(:B2,"T"."IN_STOCK")='Y' OR NVL(:B2,"T"."IN_STOCK")='N') AND (NVL(:B3,"T"."DISCOUNTED")='Y'
OR NVL(:B3,"T"."DISCOUNTED")='N'))
The classic Tom Kyte/Bryn Llewellyn approach is to generate different SQL depending on whether the parameter is null or not null, but still binding each parameter exactly once. This way will produce multiple different cursors, but maximum 2 * the number of parameter values, and it's neat and efficient. The idea is that for each parameter value, you generate either
where t.column = :b1
if :b1 has a value, or else
where (1=1 or :b1 is null)
if it's null. You could logically skip the 1=1 part, but it takes advantage of some short-circuiting logic in the Oracle SQL parser that means it won't evaluate the or condition at all because it knows there is no need. For example,
select dummy from dual where 1=1 or sqrt(-1) > 1/0;
which returns 'X' without evaluating the impossible sqrt(-1) or 1/0 expressions.
Using this approach, your SQL would be generated as something like this:
v_sql := '
select * from iavw_inst_json_table i
where (1=1 or i.iainst_natn_code = :1)
and i.iainst_regn_code = :2
and i.rank_code = :3
and (1=1 or i.rank_year = :4)
';
You could use a procedure to generate the parameter handling SQL:
declare
l_report_sql clob := 'select * from bigtable t where 1=1';
l_product_type bigtable.product_type%type;
l_in_stock bigtable.in_stock%type := 'Y';
l_discounted bigtable.discounted%type := 'N';
procedure apply_bind
( p_bind# in number
, p_column_name in varchar2
, p_value_is_null in boolean
, p_sql in out clob )
is
begin
p_sql := p_sql || chr(10) || 'and ' ||
case
when p_value_is_null then '(1=1 or :'||p_bind#||' is null)'
else p_column_name||' = :'||p_bind#
end;
end;
begin
apply_bind(1, 't.product_type', l_product_type is null, l_report_sql);
apply_bind(2, 't.in_stock', l_in_stock is null, l_report_sql);
apply_bind(3, 't.discounted', l_discounted is null, l_report_sql);
dbms_output.put_line(l_report_sql);
open :results for l_report_sql using l_product_type, l_in_stock, l_discounted;
end;
My example gives:
select * from bigtable t where 1=1
and (1=1 or :1 is null)
and t.in_stock = :2
and t.discounted = :3