This might sound like a dumb question - apologies, I'm new to SQL Server and I just want to confirm my understanding.
I've got a query that is aggregating values in a table as a subquery in different ways for different columns, e.g. for a transaction on a given day, transactions in the previous month, previous 6 months, before that, after that.
I aliased the main table as tx, then the subquery alias as tx1 so I could use for example:
tx1.TransactionDate < tx.TransactionDate
I created one column, copied it and amended the WHERE conditions.
I assumed that the scope of an alias in the subquery is bound to that subquery, so it didn't matter that the alias was the same in each case.
It seems to work, but then as neither the main table tx is altered nor the subquery tables tx1 I wouldn't know if the scope of the alias tx1 was bound to each subquery or if the initial tx1 was being reused.
Am I correct in my assumption?
Query:
SELECT tr.transaction_value ,
Isnull(
(
SELECT Sum(tr1.transaction_value)
FROM [MyDB].[dbo].[Transactions] tr1
WHERE tr1.client_ref = tr.client_ref),0)
and tr1.transaction_date > tr.transaction_date ),0) AS 'Future_Transactions' ,isnull(
(
SELECT sum(tr1.transaction_value)
FROM [MyDB].[dbo].[Transactions] tr1
WHERE tr1.client_ref = tr.client_ref),0)
AND
tr1.transaction_date < tr.transaction_date ),0) AS 'Prior_Transactions' FROM [MyDB].[dbo].[Transactions]
I think that following script can explain you everything.
SELECT 1,1,GETDATE()
INSERT INTO #t ( Id, UserId, TranDate )
SELECT 2,1,GETDATE()
INSERT INTO #t ( Id, UserId, TranDate )
SELECT 3,1,GETDATE()
SELECT tx.Id/*main alias*/,
tx1.Id /*First subquery alias*/,
tx2.Id /*Second subquery alias*/,
(SELECT Id FROM #t txs /*alias only in this one subquery/must be different from main if you want use main alias in it...*/
WHERE txs.Id = tx.Id+2 /*here is used main value = subquery value+2*/) AS Id
FROM #t tx /*main*/
JOIN (SELECT *
FROM #t tx
WHERE tx.Id = 1 /*this one using subquery values + you are not able to use here main value*/
) tx1 --alias of subquery
ON tx.Id = tx1.Id /*here is used main value = subquery value*/
CROSS APPLY (SELECT TOP 1 *
FROM #t txc /*This one must be different from main if you want use it to comparison with main*/
WHERE txc.Id > tx.Id /*this one using subquery value > main value*/
) tx2 --alias of subquery
WHERE tx.Id = 1 AND /*Subquery alias canot reference on First subquery value*/
tx1.Id = 1 AND/*Subquery alias*/
tx2.Id = 2 /*Subquery alias*/
It means that yea, it could be reused, but only if you dont want compare main / sub, because if you reuse it and for example you try to do folowing statement in subquery tx.Id > tx.Id It causes that only values in subquery will be compared. In our example it causes that you dont get anything because you comaring values in same row...
Related
Link to demo code and data: http://sqlfiddle.com/#!18/5811b/9
This while loop iterates over data and inserts data based on a merge condition, which is (roughly) "find the MergeRecordId in the earliest row that matches the ActionId, ParentActionId and Category, then insert relevant rows that have this Merge Id, using the current row's Merge Id". It ignores rows that have no "predecessors". Idx and OriginalIdx are helper columns.
Is it possible to rewrite this using a recursive CTE? Or should I be using a different technique?
This is currently what I have, but it obviously doesn't work because it doesn't match the earliest item (iRank = 1 over the Date):
; WITH cte AS
(
SELECT Idx
, ActionId
, ParentActionId
, Category
, ActionDateUpdated
, MergeRecordId
, OriginalIdx
FROM #Example
UNION ALL
SELECT -ex1.Idx
, ex2.ActionId
, ex2.ParentActionId
, ex2.Category
, ex1.ActionDateUpdated
, ex1.MergeRecordId
, ex2.Idx
FROM #Example ex1
JOIN cte ex2
ON ex1.Category = ex2.Category
AND ex1.ActionDateUpdated > ex2.ActionDateUpdated
WHERE EXISTS
(
SELECT 1
FROM #Example ex3
WHERE ex1.ActionId = ex3.ActionId
AND ex1.ParentActionId = ex3.ParentActionId
AND ex1.Category = ex3.Category
AND ex1.ActionDateUpdated > ex3.ActionDateUpdated
)
)
SELECT *
FROM cte ORDER BY ABS(idx), ABS(OriginalIdx);
Under usual circumstances it should be trivial to get the required MergeRecordId using a sub-query, but you can't use sub-queries containing CTEs in a recursive CTE. Without this filter it makes the query useless with large datasets.
(Another quirk is the deletion of the current row in the loop, but I'm not too concerned about that.)
I have a requirement where I have to check if the record for the business date already exists in the table then I need to update the values for that business date from the select statement otherwise I have to insert for that business date from the select statement. Below is my full query where I am only inserting at the moment:
INSERT INTO
gstl_calculated_daily_fee(business_date,fee_type,fee_total,range_id,total_band_count)
select
#tlf_business_date,
'FEE_LOCAL_CARD',
SUM(C.settlement_fees),
C.range_id,
Count(1)
From
(
select
*
from
(
select
rowNumber = #previous_mada_switch_fee_volume_based_count + (ROW_NUMBER() OVER(PARTITION BY DATEPART(MONTH, x_datetime) ORDER BY x_datetime)),
tt.x_datetime
from gstl_trans_temp tt where (message_type_mapping = '0220') and card_type ='GEIDP1' and response_code IN('00','10','11') and tran_amount_req >= 5000 AND merchant_type NOT IN(5542,5541,4829)
) A
CROSS APPLY
(
select
rtt.settlement_fees,
rtt.range_id
From gstl_mada_local_switch_fee_volume_based rtt
where A.rowNumber >= rtt.range_start
AND (A.rowNumber <= rtt.range_end OR rtt.range_end IS NULL)
) B
) C
group by CAST(C.x_datetime AS DATE),C.range_id
I have tried to use the if exists but could not fit in the above full query.
if exists (select
business_date
from gstl_calculated_daily_fee
where
business_date = #tlf_business_date)
UPDATE gstl_calculated_daily_fee
SET fee_total = #total_mada_local_switch_fee_low
WHERE fee_type = 'FEE_LOCAL_CARD'
AND business_date = #tlf_business_date
else
INSERT INTO
Please help.
You need a MERGE statement with a join.
Basically, our issue with MERGE is going to be that we only want to merge against a subset of the target table. To do this, we pre-filter the table as a CTE. We can also put the source table as a CTE.
Be very careful when you write MERGE when using a CTE. You must make sure you fully filter the target within the CTE to what rows you want to merge against, and then match the rows using ON
;with source as (
select
business_date = #tlf_business_date,
fee_total = SUM(C.settlement_fees),
C.range_id,
total_band_count = Count(1)
From
(
select
rowNumber = #previous_mada_switch_fee_volume_based_count + (ROW_NUMBER() OVER(PARTITION BY DATEPART(MONTH, x_datetime) ORDER BY x_datetime)),
tt.x_datetime
from gstl_trans_temp tt where (message_type_mapping = '0220') and card_type ='GEIDP1' and response_code IN('00','10','11') and tran_amount_req >= 5000 AND merchant_type NOT IN(5542,5541,4829)
) A
CROSS APPLY
(
select
rtt.settlement_fees,
rtt.range_id
From gstl_mada_local_switch_fee_volume_based rtt
where A.rowNumber >= rtt.range_start
AND (A.rowNumber <= rtt.range_end OR rtt.range_end IS NULL)
) B
group by CAST(A.x_datetime AS DATE), B.range_id
),
target as (
select
business_date,fee_type,fee_total,range_id,total_band_count
from gstl_calculated_daily_fee
where business_date = #tlf_business_date AND fee_type = 'FEE_LOCAL_CARD'
)
MERGE INTO target t
USING source s
ON t.business_date = s.business_date AND t.range_id = s.range_id
WHEN NOT MATCHED BY TARGET THEN INSERT
(business_date,fee_type,fee_total,range_id,total_band_count)
VALUES
(s.business_date,'FEE_LOCAL_CARD', s.fee_total, s.range_id, s.total_band_count)
WHEN MATCHED THEN UPDATE SET
fee_total = #total_mada_local_switch_fee_low
;
The way a MERGE statement works, is that it basically does a FULL JOIN between the source and target tables, using the ON clause to match. It then applies various conditions to the resulting join and executes statements based on them.
There are three possible conditions you can do:
WHEN MATCHED THEN
WHEN NOT MATCHED [BY TARGET] THEN
WHEN NOT MATCHED BY SOURCE THEN
And three possible statements, all of which refer to the target table: UPDATE, INSERT, DELETE (not all are applicable in all cases obviously).
A common problem is that we would only want to consider a subset of a target table. There a number of possible solutions to this:
We could filter the matching inside the WHEN MATCHED clause e.g. WHEN MATCHED AND target.somefilter = #somefilter. This can often cause a full table scan though.
Instead, we put the filtered target table inside a CTE, and then MERGE into that. The CTE must follow Updatable View rules. We must also select all columns we wish to insert or update to. But we must make sure we are fully filtering the target, otherwise if we issue a DELETE then all rows in the target table will get deleted.
How can I update the following column "LowestFinishDate" in my #temp table to hold the absolute mininium value of columns ["Finished_OldDate", "Finished_NewDate" and "Current_FinishedDate"]?
This is what the table looks like for Parent0000:
So in this case I want all 7 rows in #temp.[LowestFinishDate] for Parent0000 to be updated to the lowest date which is:
2020-11-25 14:15.
I have tried doing a CROSS/OUTER APPLY and use a table-value constructor but for some reason each LowestFinishDate rows gets updated with the correspondent value of Current_FinishedDate.
Thanks in advance
In SQL Server, I would be inclined to write this as:
with toupdate as (
select t.*,
min(least_date) over (partition by t.parentid) as new_lowestfinishdate
from #temp t cross apply
(select min(dte) as least_date
from (values (t.Finished_OldDate),
(t.Finished_NewDate)
(t.Current_FinishedDate)
) v(dte)
) v
)
update toudpate
set lowestfinishdate = new_lowestfinishdate;
The cross apply takes the minimum value of the dates within each row. The window function then takes the minimum across all rows for the parent id.
One method is use use MIN with a subquery and a VALUES table construct:
UPDATE YT
SET LowestFinishedDate = (SELECT MIN(V.FinishDate)
FROM (VALUES (YT.FinishedOldDate),
(YT.FinishedNewDate),
(YT.CurrentFinishDate)) V(FinishDate)
WHERE V.FinishDate IS NOT NULL)
FROM dbo.YourTable;
I seem to understand that Join is preferred to sub-select.
I'm unable to see how to turn the 3 sub-selects to joins.
My sub-selects fetch the first row only
I'm perfectly willing to leave this alone if it is not offensive SQL.
This is my query, and yes, those really are the table and column names
select x1.*, x2.KTNR, x3.J6NQ
from
(select D0HONB as HONB, D0HHNB as HHNB,
(
select DHHHNB
from ECDHREP
where DHAOEQ = D0ATEQ and DHJRCD = D0KNCD
order by DHEJDT desc
FETCH FIRST 1 ROW ONLY
) as STC_HHNB,
(
select FIQ9NB
from DCFIREP
where FIQ7NB = D0Q7NB
AND FIBAEQ = D0ATEQ
and FISQCD = D0KNCD
and FIGZSZ in ('POS', 'ACT', 'MAN', 'HLD')
order by FIYCNB desc
FETCH FIRST 1 ROW ONLY
) as BL_Q9NB,
(
select AAKPNR
from C1AACPP
where AACEEQ = D0ATEQ and AARCCE = D0KNCD and AARDCE = D0KOCD
order by AAHMDT desc, AANENO desc
FETCH FIRST 1 ROW ONLY
) as NULL_KPNR
from ECD0REP
) as x1
left outer join (
select AAKPNR as null_kpnr, max(ABKTNR) as KTNR
from C1AACPP
left outer join C1ABCPP on AAKPNR = ABKPNR
group by AAKPNR
) as X2 on x1.NULL_KPNR = x2.null_KPNR
left outer join (
select ACKPNR as KPNR, count(*) as J6NQ
from C1ACCPP
WHERE ACJNDD = 'Y'
group by ACKPNR
) as X3 on x1.NULL_KPNR = x3.KPNR
You've got a combination of correlated subselects and nested table expressions (NTE).
Personally, I'd call it offensive if I had to maintain it. ;)
Consider common table expressions & joins...without your data and tabvle structure, I can't give you the real statement, but the general form would look like
with
STC_HHNB as (
select DHHHNB, DHAOEQ, DHJRCD, DHEJDT
from ECDHREP )
, BL_Q9NB as ( <....>
where FIGZSZ in ('POS', 'ACT', 'MAN', 'HLD'))
<...>
select <...>
from stc_hhb
join blq9nb on <...>
Two important reasons to favor CTE over NTE...the results of a CTE can be reused Also it's easy to build a statement with CTE's incrementally.
By re-used, I mean you can have
with
cte1 as (<...>)
, cte2 as (select <...> from cte1 join <...>)
, cte3 as (select <...> from cte1 join <...>)
, cte4 as (select <...> from cte2 join cte3 on <...>)
select * from cte4;
The optimizer can choose to build a temporary results set for cte1 and use it multiple times. From a building standpoint, you can see I'm builing on each preceding cte.
Here's a good article
https://www.mcpressonline.com/programming/sql/simplify-sql-qwithq-common-table-expressions
Edit
Let's dig into your first correlated sub-query.
select D0HONB as HONB, D0HHNB as HHNB,
(
select DHHHNB
from ECDHREP
where DHAOEQ = D0ATEQ and DHJRCD = D0KNCD
order by DHEJDT desc
FETCH FIRST 1 ROW ONLY
) as STC_HHNB
from ECD0REP
What you asking the DB to do is for every row read in ECD0REP, go out and get a row from ECDHREP. If you're unlucky, the DB will have to read lots of records in ECDHREP to find that one row. Generally, consider that with correlated sub-query the inner query would need to read every row. So if there's M rows in the outer and N rows in the inner...then you're looking at MxN rows being read.
I've seen this before, especially on the IBM i. As that's how an RPG developer would do it
read ECD0REP;
dow not %eof(ECD0REP);
//need to get DHHHNB from ECDHREP
chain (D0ATEQ, D0KNCD) ECDHREP;
STC_HHNB = DHHHNB;
read ECD0REP;
enddo;
But that's not the way to do it in SQL. SQL is (supposed to be) set based.
So what you need to do is think of how to select the set of records out of ECDHREP that will match up to the set of record you want from ECD0REP.
with cte1 as (
select DHHHNB, DHAOEQ, DHJRCD
from ECDHREP
)
select D0HONB as HONB
, D0HHNB as HHNB
, DHHHBN as STC_HHNB
from ECD0REP join cte1
on DHAOEQ = D0ATEQ and DHJRCD = D0KNCD
Now maybe that's not quite correct. Perhaps there's multiple rows in ECDHREP with the same values (DHAOEQ, DHJRCD); thus you needed the FETCH FIRST in your correlated sub-query. Fine you can focus on the CTE and figure out what needs to be done to get that 1 row you want. Perhaps MAX(DHHHNB) or MIN(DHHHNB) would work. If nothing else, you could use ROW_NUMBER() to pick out just one row...
with cte1 as (
select DHHHNB, DHAOEQ, DHJRCD
, row_number() over(partition by DHAOEQ, DHJRCD
order by DHAOEQ, DHJRCD)
as rowNbr
from ECDHREP
), cte2 as (
select DHHHNB, DHAOEQ, DHJRCD
from cte1
where rowNbr = 1
)
select D0HONB as HONB
, D0HHNB as HHNB
, DHHHBN as STC_HHNB
from ECD0REP join cte2
on DHAOEQ = D0ATEQ and DHJRCD = D0KNCD
Now you're dealing with sets of records, joining them together for your final results.
Worse case, the DB has to read M + N records.
It's not really about performance, it's about thinking in sets.
Sure with a simple statement using a correlated sub-query, the optimizer will probably be able to re-write it into a join.
But it's best to write the best code you can, rather then hope the optimizer can correct it.
I've seen and rewritten queries with 100's of correlated & regular sub-queries....in fact I've seen a query that had to be broken into 2 because there were two many sub-queries. The DB has a limit of 256 per statement.
I'm going to have to differ with Charles here if the FETCH FIRST 1 ROW ONLY clauses are necessary. In this case you likely can't pull those sub-selects out into a CTE because that CTE would only have a single row in it. I suspect you could pull the outer sub-select into a CTE, but you would still need the sub-selects in the CTE. Since there appears to be no sharing, I would call this personal preference. BTW, I don't think pulling the sub-selects into a join will work for you either, in this case, for the same reason.
What is the difference between a sub-select and a CTE?
with mycte as (
select field1, field2
from mytable
where somecondition = true)
select *
from mycte
vs.
select *
from (select field1, field2
from mytable
where somecondition = true) a
It's really just a personal preference, though depending on the specific requirements, a CTE can be used multiple times within the SQL statement, but a sub-select will be more correct in other cases like the FETCT FIRST clause in your question.
EDIT
Let's look at the first sub-query. With the appropriate index:
(
select DHHHNB
from ECDHREP
where DHAOEQ = D0ATEQ and DHJRCD = D0KNCD
order by DHEJDT desc
FETCH FIRST 1 ROW ONLY
) as STC_HHNB,
only has to read one record per row in the output set. I don't think that is terribly onerous. This is the same for the third correlated sub-query as well.
That index on the first correlated sub-query would be:
create index ECDHREP_X1
on ECDHREP (DHAOEQ, DHJRCD, DHEJDT);
The second correlated sub-query might need more than one read per row, just because of the IN predicate, but it is far from needing a full table scan.
My table looks like this:
Value Previous Next
37 NULL 42
42 37 3
3 42 79
79 3 NULL
Except, that the table is all out of order. (There are no duplicates, so that is not an issue.) I was wondering if there was any way to make a query that would order the output, basically saying "Next row 'value' = this row 'next'" as it's shown above ?
I have no control over the database and how this data is stored. I am just trying to retrieve it and organize it. SQL Server I believe 2008.
I realize that this wouldn't be difficult to reorganize afterwards, but I was just curious if I could write a query that just did that out of the box so I wouldn't have to worry about it.
This should do what you need:
WITH CTE AS (
SELECT YourTable.*, 0 Depth
FROM YourTable
WHERE Previous IS NULL
UNION ALL
SELECT YourTable.*, Depth + 1
FROM YourTable JOIN CTE
ON YourTable.Value = CTE.Next
)
SELECT * FROM CTE
ORDER BY Depth;
[SQL Fiddle] (Referential integrity and indexes omitted for brevity.)
We use a recursive common table expression (CTE) to travel from the head of the list (WHERE Previous IS NULL) to the trailing nodes (ON YourTable.Value = CTE.Next) and at the same time memorize the depth of the recursion that was needed to reach the current node (in Depth).
In the end, we simply sort by the depth of recursion that was needed to reach each of the nodes (ORDER BY Depth).
Use a recursive query, with the one i list here you can have multiple paths along your linked list:
with cte (Value, Previous, Next, Level)
as
(
select Value, Previous, Next, 0 as Level
from data
where Previous is null
union all
select d.Value, d.Previous, d.Next, Level + 1
from data d
inner join cte c on d.Previous = c.Value
)
select * from cte
fiddle here
If you are using Oracle, try Starts with- connect by
select ... start with initial-condition connect by
nocycle recursive-condition;
EDIT: For SQL-Server, use WITH syntax as below:
WITH rec(value, previous, next) AS
(SELECT value, previous, next
FROM table1
WHERE previous is null
UNION ALL
SELECT nextRec.value, nextRec.previous, nextRec.next
FROM table1 as nextRec, rec
WHERE rec.next = nextRec.value)
SELECT value, previous, next FROM rec;
One way to do this is with a join:
select t.*
from t left outer join
t tnext
on t.next = tnext.val
order by tnext.value
However, won't this do?
select t.*
from t
order by t.next
Something like this should work:
With Parent As (
Select
Value,
Previous,
Next
From
table
Where
Previous Is Null
Union All
Select
t.Value,
t.Previous,
t.Next
From
table t
Inner Join
Parent
On Parent.Next = t.Value
)
Select
*
From
Parent
Example