i have this query
update t_reconcile_biller b set status = case when exists (
SELECT i.credit_amount, i.recipt_no,
i.credit_acct_no, i.recon_date
FROM t_reconcile_t24 i
WHERE i.debit_acct_no = 'IDR1720600010001'
and b.col_4 = i.credit_amount
AND b.col_7 = i.recipt_no
AND b.col_2 = i.card_no
AND i.status <> 5
) then 1 else 2 end
where b.status <> 5 AND b.file_name in ('20200923-bnf', '20200922-bnf')
AND b.col_13 = 'DESTIONATION' AND b.col_4 <> '0'
AND b.col_14 in ('A', 'FC')
which take 52 second to run with postgres, where t_reconcile_biller has 3.900 rows and t_reconcile_t24 has 32.000 rows.
How can i make this query faster and more efficient ?
The following index on the table appearing inside the exists clause might help:
CREATE INDEX rec_idx ON t_reconcile_t24 (
credit_amount, recipt_no, card_no, status );
If used, Postgres could use the above index to quickly lookup each record in the target table. In addition, the following index on the outer table might also be helpful:
CREATE INDEX biller_idx ON t_reconcile_biller (
status, file_name, col_4, col_13, col_14 );
Related
The following query takes nearly 1 minute to execute. The table has 200k records. Can it be rewritten to perform faster?
SELECT CLIENT_FNM FROM (SELECT DISTINCT TRIM(A.CLIENT_FNM) AS client_fnm
FROM gen_clientvendor_m A JOIN gen_clientvendor_m B ON (TRIM(A.CLIENT_FNM) = TRIM(B.CLIENT_FNM))
WHERE A.CLIENT_VENDOR_ID <> B.CLIENT_VENDOR_ID
AND A.PARTY_TYPE = 'C'
AND A.status = 'A'
ORDER BY TRIM(CLIENT_FNM))
WHERE ROWNUM <= 5
I'd aggregate over TRIM(client_fnm), so as to read the table only once.
select client_fnm
from
(
select
trim(client_fnm) as client_fnm,
row_number() over (order by trim(client_fnm)) as rn
from gen_clientvendor_m
group by trim(client_fnm)
having count(distinct client_vendor_id) > 1
and count(case when party_type = 'C' and status = 'A' then 1 end) > 0
)
where rn <= 5;
I've replaced Oracle's ROWNUM with starndard SQL's ROW_NUMBER. (Better would be FETCH FIRST n ROWS ONLY, but that's only available as of Oracle 12c.)
To get this fast you should have the following function index:
create index idx1 on gen_clientvendor_m(trim(client_fnm));
or even a covering index:
create index idx2 on gen_clientvendor_m(trim(client_fnm), party_type, status, client_vendor_id);
Try this:
SELECT a.CLIENT_FNM
FROM (select distinct TRIM(CLIENT_FNM)CLIENT_FNM, CLIENT_VENDOR_ID, PARTY_TYPE, status from gen_clientvendor_m) A
JOIN gen_clientvendor_m B ON A.CLIENT_FNM = TRIM(B.CLIENT_FNM)
WHERE A.CLIENT_VENDOR_ID <> B.CLIENT_VENDOR_ID
AND A.PARTY_TYPE = 'C'
AND A.status = 'A'
ORDER BY TRIM(CLIENT_FNM))
WHERE ROWNUM <= 5
For your query I'd provide the following two indexes:
create index index1 on gen_clientvendor_m(party_type, status, trim(client_fnm), client_vendor_id);
to find the records with PARTY_TYPE = 'C' AND status = 'A' quickly and provide trim(client_fnm) and client_vendor_id for the join.
And
create index index2 on gen_clientvendor_m(trim(client_fnm), client_vendor_id);
to quickly self-join on these two fields.
I have a scenario where I have to search value of column 1 in first table to see whether it matches some value in another table.
This should continue in a loop until the last row on first table has been compared.
No loops needed. You can do this easily as a set based operation using exists()
select *
from FirstTable
where exists (
select 1
from SecondTable
where FirstTable.Column1 = SecondTable.Column1
);
To find the opposite, where the row in the first table does not have a match based on Column1, you can use not exists()
select *
from FirstTable
where not exists (
select 1
from SecondTable
where FirstTable.Column1 = SecondTable.Column1
);
If you want to identify which rows have a match and don't you can use:
select FirstTable.*
, MatchFound = case when x.Column1 is null then 'No' else 'Yes' end
, x.Column1
from FirstTable
outer apply (
select top 1
*
from SecondTable
where FirstTable.Column1 = SecondTable.Column1
) as x
Say I have a table t with 2 columns:
a int
b int
I can do a query such as:
select b
from t
where b > a
and a in(1,2,3)
order by b
where 1,2,3 is provided from the outside.
Obviously, the query can return no rows. In that case, I'd like to select everything as if the query did not have the and a in(1,2,3) part. That is, I'd like:
if exists (
select b
from t
where b > a
and a in(1,2,3)
)
select b
from t
where b > a
and a in(1,2,3)
order by b
else
select b
from t
where b > a
order by b
Is there a way to do this:
Without running two queries (one for exists, the other one the actual query)
That is less verbose than repeating queries (real queries are quite long, so DRY and all that stuff)
Using NOT EXISTS with a Sub Query to Determine if condition exists
SELECT b
FROM
t
WHERE
b > a
AND (
NOT EXISTS (SELECT 1 FROM #Table WHERE a IN (1,2,3))
OR a IN (1,2,3)
)
ORDER BY
b
The reason this works is because if the condition exists then the OR statement will include the rows and if the condition does not exist then the NOT EXISTS will include ALL rows.
Or With Common Table Expression and window Function with Conditional Aggregation.
WITH cte AS (
SELECT
b
,CASE WHEN a IN (1,2,3) THEN 1 ELSE 0 END as MeetsCondition
,COUNT(CASE WHEN a IN (1,2,3) THEN a END) OVER () as ConditionCount
FROM
t
)
SELECT
b
FROM
cte
WHERE
(ConditionCount > 0 AND MeetsCondition = 1)
OR (ConditionCount = 0)
ORDER BY
b
I find it a bit "ugly". Maybe it would be better to materialize output from your query within a temp table and then based on count from temp table perform first or second query (this limits accessing the original table from 3 times to 2 and you will be able to add some flag for qualifying rows for your condition not to repeat it). Other than that, read below . . .
Though, bear in mind that EXISTS query should execute pretty fast. It stops whether it finds any row that satisfies the condition.
You could achieve this using UNION ALL to combine resultset from constrained query and full query without constraint on a column and then decide what to show depending on output from first query using CASE statement.
How CASE statement works: when any row from constrained part of your query is found, return resultset from constrainted query else return everything omitting the constraint.
If your database supports using CTE use this solution:
with tmp_data as (
select *
from (
select 'constraint' as type, b
from t
where b > a
and a in (1,2,3) -- here goes your constraint
union all
select 'full query' as type, b
from t
where b > a
) foo
)
SELECT b
FROM tmp_data
WHERE
CASE WHEN (select count(*) from tmp_data where type = 'constraint') > 0
THEN type = 'constraint'
ELSE type = 'full query'
END
;
HI Gurus,
I'm looking to replace an IN clause with exists, but despite reading other similar cases on here I've not been able to apply them to my dataset.
I am looking to add in a column to my main query which tells me if a fund is found within a separate list, and if it does then label it 'emergency' and if not then 'non-emergency'
The list is defined like so:
select
f.id
FROM _audit a
INNER JOIN _fund f ON a.article_id = f.id
WHERE a.entity_name = 'Fund'
AND a.Changes LIKE
'%finance_code2%OldValue>3%'
)
UNION
(
select
id AS fund_reference
FROM _fund
WHERE (finance_code2 LIKE '3%'
OR finance_code2 LIKE '9%')
AND finance_code2 IS NOT NULL
And so what I am looking for is essentially something like:
SELECT
...Main query here...
,CASE WHEN fund_id IN (list_details) THEN 'emergency' else 'non-emergency' end
I know that it would be more efficient to do something like
SELECT
...Main query here...
,SELECT CASE WHEN EXISTS
(SELECT fund_id FROM list_details WHERE fund_id IS NOT NULL) THEN 'emergency' else 'non-emergency' END
But every time I try it keeps returning false values (saying that funds are contained within the list when they are not)
In case it helps I'm using sql server 2005 and the main query is listed below, where the list_details result (id) is joined onto donation_fund_allocation on list_details.id = donation_fund_allocation.fund_id
As always any clue would be massively appreciated :)
Thanks!
Main query
SELECT
don.supporter_id AS contact_id
,don.id AS gift_id
,YEAR(don.date_received) AS calendar_year
,YEAR(don.date_received) - CASE WHEN MONTH(don.date_received) < 4 THEN 1 ELSE 0 END AS financial_year
,don.date_received AS date_received
,don.event_id AS event_id
,SUM(CASE WHEN don.gift_aid_status <> 4 THEN don.value_gross * ((dfa.percentage) / 100)
WHEN don.gift_aid_status = 4 AND don.value_net > don.value_gross
AND don.value_net <> 0 THEN don.value_net * ((dfa.percentage) / 100)
ELSE don.value_gross * ((dfa.percentage) / 100)
END
) AS donation_value
--**List details query to go in here**
FROM donation don WITH (nolock)
INNER JOIN donation_fund_allocation dfa WITH (nolock) ON dfa.donation_id = don.id
WHERE don.supporter_id IS NOT NULL
AND don.status = 4
AND don.value_gross <> 0
GROUP BY don.supporter_id
,don.id
,don.date_received
,don.event_id
You need to correlate the exists call with the outer query. As written you are just asking if there exist any rows in list_details where fund_id isn't null
So, what you actually want is
SELECT
...Main query here...
,SELECT CASE WHEN EXISTS
(SELECT 1 FROM list_details WHERE fund_id = outer.fund_id) THEN 'emergency' else 'non-emergency' END
Where outer is the table alias for where fund_id can be found in your main select
You could write a function which takes the fund_id and returns an appropriate string value of "emergency" or "non-emergency".
I'm trying to create a query that would generate a cross-check table with about 40 custom columns that show Y or N. Right now I have
SELECT DISTINCT [Company],
[Option1],
[Option2],
[Option3],
CASE
WHEN [Table1].[ID1] IN (SELECT ID2 FROM Table2 WHERE Variable = 1 AND Bit = 1) THEN
'Y'
ELSE 'N'
END AS 'CustomColumn1:',
CASE
WHEN [Table1].[ID1] IN (SELECT ID2 FROM Table2 WHERE Variable = 2 AND Bit = 1) THEN
'Y'
ELSE 'N'
END AS 'CustomColumn1:',
CASE
WHEN [Table1].[ID1] IN (SELECT ID2 FROM Table2 WHERE Variable = 3 AND Bit = 1) THEN
'Y'
ELSE 'N'
END AS 'CustomColumn1:',
.............
-- REPEAT ANOTHER 40 times
FROM [Table1]
WHERE [Table1].[OtherCondition] = 'True'
ORDER BY [Company]
So my question is, how do I create a loop (while? for?) that will loop on variable and assign Y or N to the row based on the condition, rather than creating 40+ Case statements?
You couldn't use a loop, but you could create a stored procedure/function to perform the sub-select and case expression and call that 40 times.
Also, you could improve performance of the sub-select by changing it to
SELECT 1 FROM Table2 WHERE EXISTS [Table2].[ID2] = [Table1.ID1] AND Variable = 3 AND Bit = 1
A loop (that is, iterating through a cursor) works on rows, not columns. You will still have to have 40 expressions, one for each column, and the performance will be terrible.
Let SQL Server do its job. And do your bit by telling exactly what you need and creating proper indices. That is, replace
CASE WHEN [Table1].[ID1] IN (SELECT ID2 FROM Table2 WHERE Variable = 2 AND Bit = 1)
with
CASE WHEN EXISTS (SELECT 0 FROM Table2 WHERE ID2 = [Table1].[ID1] AND Variable = 2 AND Bit = 1)
If the output is so vastly different than the schema, there is a question as to whether the schema properly models the business requirements. That said, I would recommend just writing the SQL. You can simplify the SQL like so:
Select Company
, Option1, Option2, Option3
, Case When T2.Variable = 1 Then 'Y' Else 'N' End As CustomCol1
, Case When T2.Variable = 2 Then 'Y' Else 'N' End As CustomCol2
, Case When T2.Variable = 3 Then 'Y' Else 'N' End As CustomCol3
, Case When T2.Variable = 4 Then 'Y' Else 'N' End As CustomCol4
...
From Table1 As T1
Left Join Table2 As T2
On T2.ID2 = T1.ID
And T2.Bit = 1
Where T1.OtherCondition = 'True'
Group By T1.Company
Order By T1.Company
If you want to write something that can help you auto-gen those Case statements (and you are using SQL Server 2005+), you could do something like:
With Numbers As
(
Select 0 As Value
Union All
Select Value + 1
From Numbers
Where Value < 41
)
Select ', Case When T2.Variable = ' + Cast(N.Value As varchar(10)) + ' Then ''Y'' Else ''N'' End As CustomCol' + Cast(N.Value As varchar(10))
From Numbers As N
You would run the query and copy and paste the results into your procedure or code.
One way could have been to use Pivot statement, which is in MS SQL 2005+. But even in that you have to put 1 ... 40 hardcoded columns in pivot statement.
Other way i can think of is to create dynamic SQL, but it is not so much recommended, So what we can do is we can create a dynamic sql query by running a while loop on table and can create the big sql and then we can execute it by using sp_execute. So steps would be.
int #loopVar
SET #loopVar = 0
int #rowCount
varchar #SQL
SET #SQl = ''
Select #rowcount = Count(ID2) from Table2
WHILE(#loopVar <= #rowCount)
BEGIN
// create ur SQL here
END
sp_execute(#SQL)