Select with filter on function parameters, if specified - sql

I have a function to try to match partial data to a database row.
I want it to find a match if the parameter is non null; if it's null it should ignore that parameter.
If one of the parameters has a value but finds no match, the query returns no rows.
In pseudocode, that's pretty much how it'd go:
get all rows where:
param_a matches col_a when param_b is not null else don't check this column
AND param_b matches col_b when param_b is not null else don't check this column
AND param_c matches col_c when param_c is not null else don't check this column
AND param_d matches col_d when param_d is not null else don't check this column
AND param_e matches col_e when param_e is not null else don't check this column
What I do right now:
SELECT * FROM table
WHERE nvl(param_a, col_a) = col_a
AND nvl(param_b, col_b) = col_b
AND nvl(param_c, col_c) = col_c
AND nvl(param_d, col_d) = col_d;
Etc... It works, but I'm not sure it's the best option. A colleague suggested that I use
SELECT * FROM table
WHERE (param_a = col_a or param_a is null)
AND (param_b = col_b or param_b is null)
AND (param_c = col_c or param_c is null)
AND (param_d = col_d or param_d is null);
As this is used for an auto-complete feature in a web application, the query is executed a lot, as the user types. Being fast is essential. The strictest columns are filtered first to reduce the number of rows to process.
Is either of these options preferable? If not, how would you do it?
EDIT:
I wrote the question to be generic, but to put it in context: in this case it's for addresses. param_a is actually postal code, param_b street name etc... The function gets the string the user writes (ex: 999 Random St, Fakestate, Countryland, 131ABD) and calls a procedure on it that tries to split it and returns a table containing address, city, country, etc... that is used by the select statement (which is the subject of the question).

I believe the second solution is better. It allows Oracle to skip evaluating colA/B/C/D is the corresponding parameter is null.
However, it would be cleaner and faster if you dynamically build the query.
For example either in sql or in a programming language you can do something like this:
whereClause = 'WHERE 1 = 1'
IF paramA is not null
then whereClause += ' AND param_a = col_a'
else if paramB is not null
then whereClause += ' AND param_b = col_b'
etc...
For indexes I would only index the commonly used column combinations. There are too many combinations to cover them all. Pick the ones that give you the most bang for your buck.

If you are trying to go at typing speed, then I would suggest the following approach. Create a separate query for each combination of parameters. This is a total of 24 queries, with the where clauses such as:
WHERE param_a = col_a
WHERE param_b = col_b
. . .
WHERE param_a = col_a and param_b = col_b
. . .
WHERE param_a = col_a and param_b = col_b and param_c = col_c and param_d = col_d
Then, precompile these twenty-four queries.
Then choose the appropriate query based on the current state of the parameters.
I would also add indexes, at a minimum:
table(param_a, param_b, param_c, param_d)
table(param_b, param_c, param_d, param_a)
table(param_c, param_d, param_a, param_b)
table(param_d, param_a, param_b, param_c)
This will at least cover all cases with one parameter. You might want to include other indexes for other parameters.

Related

Why does WHERE clause removes null values from the result?

I have a table I want to filter data from. I tried the following query
SELECT
SIS, COUNT(*)
FROM DL_SQ_DEV_INT.SMRY_DAILY_TRAILER_REPORT
GROUP BY 1;
Result:
BL,17386
EQ,3242
FIFO,5747
GR,15655
HOLD,13035
LT BL,20566
LT GR,14615
LT OR,14190
LT PU,13877
LT YE,13683
null,223376
OR,15727
PI,3563
PU,16105
RW,200
TA,6
tbd,25302
WH,1945
YE,14510
Now when I add a WHERE clause in it, it filters out the null values. The query is a not equal to (<>). How can I avoid that and still have the null values in my result? Changing null to blank or space?
SELECT
SIS, COUNT(*)
FROM DL_SQ_DEV_INT.SMRY_DAILY_TRAILER_REPORT
WHERE UPPER(TRIM(SIS)) <> 'EQ'
GROUP BY 1;
Result:
BL,17386
FIFO,5747
GR,15655
HOLD,13035
LT BL,20566
LT GR,14615
LT OR,14190
LT PU,13877
LT YE,13683
OR,15727
PI,3563
PU,16105
RW,200
TA,6
tbd,25302
WH,1945
YE,14510
Neither "not equal" nor "equal" will select a value that is NULL.
SQL uses "three-way logic" where an expression can be true or false or unknown. NULL is the absence of any value at all so it cannot be equal to something, and if it cannot be equal to a compared value it also cannot be "not equal", instead it is unknown.
To overcome this you need to treat NULL explicitly in your where clause, to include NULLs use OR SIS IS NULL
SELECT
SIS, COUNT(*)
FROM DL_SQ_DEV_INT.SMRY_DAILY_TRAILER_REPORT
WHERE UPPER(TRIM(SIS)) <> 'EQ' OR SIS IS NULL
GROUP BY 1;
You should use this:
SELECT
SIS, COUNT(*)
FROM DL_SQ_DEV_INT.SMRY_DAILY_TRAILER_REPORT
WHERE TRIM(SIS) IS NULL OR UPPER(TRIM(SIS)) <> 'EQ'
GROUP BY 1;
Why does your WHERE clause removes null values from the result: NULL value means the absence of value or value is unknown, so that you can't compare NULL value using scalar value operator, <> 'EQ' will return unknown or not true.
You could refer more post about NULL value in sql in sof, eg this link or search in gg

Where statement in my query is supposed to return only today's and yesterday's dates, but is still returning earlier dates

I am looking to only retrieve data from the past 24 hours. The WHERE statement I am using, in theory, should retrieve only from those productiondates. However, I am still having week-old productiondates returned. Any thoughts on how to improve this, or am I doing it wrong? I am using periscope.
select example1,
example2,
example3,
productiondate,
example4,
example5
from final
where exampleX = exampleY or exampleX is null
and productiondate > DATEADD(day,-1, GETDATE())
and example1 <> 'XXX'
and example2 <> 'YYY'
and example2 <> 'ZZZ'
order by 2
Logical operator precedence in SQL can be surprising. You need parentheses around the OR.
where (exampleX = exampleY or exampleX is null)
Alternatively, you could do this:
where coalesce(exampleX, exampleY) = exampleY

Include NULL values in a designated field in my where clause on SSRS

I am learning SQL so be gentle. If I have designated a specific role in my where clause it is only pulling those cases where that role is populated. How can I also include the NULL values or those roles that are blank?
Here is the where clause now:
WHERE (dbo.vcases.lawtype = 'My Cases') AND
(dbo.vcase_parties_people.role_sk = 4001) AND
**(v1.role_sk = 3940) AND
(v1.report_ind = 'Y') AND
(v2.role_sk = 3939) AND
(v2.report_ind = 'Y')** AND
(dbo.vcases.case_type NOT IN ('Case type 1', 'Case type 2'))
The COALESCE() expression in SQL is useful for substituting a default value when NULL is encountered for a given column or expression. When the query optimizer encounters a COALESCE() call, it will internally rewrite that expression to an equivalent CASE...WHEN expression. In your sample query, WHERE (COALESCE(v1.role_sk, 3940) = 3940) would operate (and optimize) the same as WHERE (CASE WHEN v1.role_sk IS NOT NULL THEN v1.role_sk ELSE 3940 END = 3940).
Since your example specifically involves a condition in the WHERE clause, you may want to use an OR operation, which could optimize better than a COALESCE() expression: WHERE (v1.role_sk = 3940 OR v1.role_sk IS NULL).
This is also assuming that any joins in your query aren't filtering out rows whose role_sk column is NULL.
You might edit your code as follows:
WHERE (dbo.vcases.lawtype = 'My Cases') AND
(dbo.vcase_parties_people.role_sk = 4001) AND
(v1.role_sk = 3940 OR v1.role_sk IS NULL) AND
(v1.report_ind = 'Y') AND
(v2.role_sk = 3939) AND
(v2.report_ind = 'Y') AND
(dbo.vcases.case_type NOT IN ('Case type 1', 'Case type 2'))
The use of the Coalesce function has been suggested but a good rule of thumb in SQL is to avoid the use of functions in the WHERE clause because it reduces the efficiency of the table's indexes. Functions in WHERE clause often cause Index-Scans instead of the more efficient Index-Seeks.

PostgreSQL: transforming a column based on column value

I want to transform col_A in my table based on the values like below:
col_A
----------------------------
Hello_axd_sdc_we_world
Hello_g_world
Hello_world
Goodbye_A
Goodbye_sdg_Sda
Goodbye
Goodbye_asd_asd_Sddg
I would like the transformed column d_col_A looks like:
col_A d_col_A
-----------------------------------------
Hello_axd_sdc_we_world Hello_world
Hello_g_world Hello_world
Hello_world Hello_world
Goodbye_A Goodbye
Goodbye_sdg_Sda Goodbye
Goodbye Goodbye
Goodbye_asd_asd_Sddg Goodbye
And here is my rules:
If col_A starts with Hello and end with World
Then d_col_A = Hello_Wolrd
If col_A starts with Goodbye
Then d_col_A = Goodbye
Is this something possible? Thanks!
Create a table called 'lookup' or something to that extent as ID,first_param,second_param,translate_value
1,hello,world,hello_world
2,goodbye,null,goodbye
Now some join fun. Postgres has a left() and right() and length() function that we can make use of here. more complicated rule sets could also include a like function (third parameter = must contain this word?) if desired.
Select a.col_A , l.translate_value
from table_a a
left join lookup_table l
on left(a.col_a,length(l.first_param)) = l.first_param
and (right(a.col_a,length(l.second_param)) = l.second_param or l.second_param is null)
I did a left join so nulls get produced when the rules aren't met rather than dropping lines. I lack a postgres test environment, but syntax should be right.
Case version
select case when left(col_a,5) = hello and right(col_a,5) = 'world' then 'hello world'
case when left(col_a,7) = 'goodbye' then goodbye
else 'no clue what this means'
from table_a

Why 'NOT IN' operator not fetching records for the columns which is not having any value?

I have a table with a column name CORP_SERIOUS,the values of this column can be 1 or 2 or empty. When I am searching unknown values records means empty field using NOT IN operator i am getting the count is zero.
Why the below query is not fetching the records for the field which doesn't have vm.CORP_SERIOUS NOT IN('1','2')?
Here is the Query:
SELECT COUNT(*)
FROM
( SELECT DISTINCT vm.vaer_no vaer,
vm.vaer_no_version version,
vm.priority priority,
vm.case_reported_in reportedIn,
vsr.originalreceivedate initialRcvDate,
vsr.mostrecentinfodate latestRcvDate,
vsr.primarysourcecountry reportingCountry,
vsr.occurcountry occurCountry,
vsrd.primary_source_full_name reporterFullName,
vsrd.sender_full_name senderFullName,
vm.record_id recordId,
vm.notes_flag notesFlag,
vm.dmi_product product,
vmf.VAERS_LINKED,
vm.VAER_STATUS status,
vm.company_unit_name companyunit,
FIND_WF_ACTIVITY(vm.record_id,JBPM_PROCESS_INST_ID),
vmf.CORRESPONDENCE_FLAG corresFlag,
vmf.READ_UNREAD_CORRESPONDENCE readUnreadCorresp,
vm.vaer_mode vmode,
srcDoc.doc_name,
vsr.seriousnessdecision serious,
vm.reportduedate reportduedate,
vm.MANUALLY_LOCKED,
vm.LOCKED_BY,
vm.LOCKED_REASON,
vm.LOCKED_DATE,
vm.vaer_delete,
vm.vaer_nullify nullify,
vm.archived,
vm.COMPLETION_FLAG,
vm.ASSIGNED_USER_GROUP,
COALESCE(vsrd.TEEATESPECIESDECODE,vsrd.SPECIESDECODE,vsrd.OTHERSPECIES,vsrd.TREATEDSPECIES),
COALESCE(vsrd.ANIMAL_BREEDDECODE,vsrd.TREATED_BREEDDECODE,vsrd.ANIMAL_BREED,vsrd.TREATED_BREED),
vm.APPROVED_VAER,
vm.SUBSTANCE_ADDED,
vm.MULTIPLEBREEDADDED,
msg_q.MDN_DATE,
vm.CASE_SOURCE,
vm.MESSAGENUMB,
vsr.NULLIFICATIONREASON,
vm.CREATED_BY,
vm.ASSESSMENT_SOURCE,
vm.ASSESSMENT_CLASSIFICATION,
vm.DMI_VEDDRA_TERM,
vsr.CASEREGISTRATIONTYPE,
vm.AUTHORIZATIONCOMPANY,
vm.E2B_DMI_PRODUCT brandname,
vm.E2B_SUBSTANCE_ADDED e2bSubstanceAdded,
vm.ACCOUNT account,
ACK.record_id ack_recId ,
vm.SUBMITTED_DELAY
FROM agvet_vae_info vm,
agvet_vae_safetyreport vsr,
agvet_vae_safetyreport_detail vsrd,
AGVET_VAE_SOURCE_DOC srcDoc,
AGVET_VAE_FLAGS vmf,
E2B_MESSAGE_QUEUE msg_q,
E2B_MESSAGE_ACK ACK
WHERE vm.fk_avsr_rec_id = vsr.record_id
AND vsr.fk_avsrd_rec_id = vsrd.record_id
AND srcdoc.record_id(+) = vm.fk_vet_source_doc_rec_id
AND vm.IMPORT_FLAG <> 1
AND (vm.DRAFT_SUBMIT_FLAG = 0
OR vm.DRAFT_SUBMIT_FLAG = 2)
AND vm.VAER_NO = vmf.VAER_NO(+)
AND vm.E2B_MESSAGE_LIST_TYPE<>01
AND vm.MESSAGENUMB = msg_q.MESSAGE_NUMBER(+)
AND vm.MESSAGENUMB = ACK.MESSAGE_NUMBER(+)
AND (vm.assigned_to = 48626
OR vm.assigned_to IS NULL
OR vm.assigned_to = 320538
OR vm.assigned_to = 320529
OR vm.assigned_to = 406699)
AND EXISTS
(SELECT *
FROM jbpm_token jt
WHERE jt.node_ IN ( 135,140,146,137,132,129,127,148,144 )
AND jt.processinstance_ = vm.jbpm_process_inst_id
OR vm.jbpm_process_inst_id IS NULL
)
AND ( fn_access_vet_products(48658,vm.RECORD_ID, vm.CASE_REPORTED_IN)=1)
AND (vm.PRIORITY IN ( 02 )
OR vm.PRIORITY IS NULL)
AND ( upper(vm.CORP_SERIOUS) NOT IN ('1','2') )
AND vm.ARCHIVED = 0
AND vm.vaer_nullify = 0
AND vm.vaer_delete = 0
)
The value NULL means unknown.
WHERE vm.CORP_SERIOUS IN('1','2')
is TRUE for '1'
is FALSE for '3'
is NULL for NULL
And
WHERE vm.CORP_SERIOUS NOT IN('1','2')
is NOT TRUE, hence FALSE for '1'
is NOT FALSE hence TRUE for '3'
is NOT NULL, which is again NULL for NULL
As NULL means unknown, we don't know whether the value (that we don't know) is in the given set or not. So the answer is "I don't know", no matter if we ask whether the value is in the list or whether the value is not in the list.
Imagine we don't know John's phone number and I show you some numbers and ask you whether John's number is among them. You can't say yes, you can't say no, you can only say maybe. Same with the DBMS. It cannot tell you TRUE or FALSE, it can only tell you NULL. Now the WHERE clause works like this: The query returns all records for which the given criteria is TRUE. NULL is not TRUE, so the records are not returned.
(Even if NULL itself were in the list, we wouldn't know whether the unknown value in the set is the same as the unknown value in the record. The WHERE clause would still result in NULL. It would make a difference for '3' though: WHERE vm.CORP_SERIOUS NOT IN('1','2', null) would suddenly result in NULL too, because the unknown value in the set could or could not be '3'.)
You could ask: Give me all unknown values plus the values in the list.
WHERE vm.CORP_SERIOUS IS NULL OR vm.CORP_SERIOUS NOT IN('1','2')
Or you could ask:
WHERE NVL(vm.CORP_SERIOUS, 'TREAT AS NOT IN THE LIST') NOT IN('1','2')
(Well, of course the string 'TREAT AS NOT IN THE LIST' must not be in the list then :-)
You can try this query
convert(varchar(50),vm.CORP_SERIOUS) NOT IN('1','2')