How to avoid Sorting in Union ALL - sql

MY question is simple, How do you avoid the automatic sorting which the UNION ALL query does?
This is my query
SELECT * INTO #TEMP1 FROM Final
SELECT * INTO #TEMP2 FROM #TEMP1 WHERE MomentId = #MomentId
SELECT * INTO #TEMP3 FROM #TEMP1 WHERE RowNum BETWEEN #StartRow AND #EndRow
SELECT * INTO #TEMP4 FROM (SELECT *FROM #TEMP3 UNION ALL SELECT *FROM #TEMP2) as tmp
SELECT DISTINCT * FROM #TEMP4
I'm using SQL Server 2008. I need the Union ALL to perform like a simple Concatenate, which it isn't! Appreciate your help in this.

I think you're mistaken on which operation is actually causing the sort. Check the code below, UNION ALL will not cause a sort. You may be looking at the DISTINCT operation, which uses a sort (it sorts all items and the eliminates duplicates)
CREATE TABLE #Temp1
(
i int
)
CREATE TABLE #temp2
(
i int
)
INSERT INTO #Temp1
SELECT 3 UNION ALL
SELECT 1 UNION ALL
SELECT 8 UNION ALL
SELECT 2
INSERT INTO #Temp2
SELECT 7 UNION ALL
SELECT 1 UNION ALL
SELECT 5 UNION ALL
SELECT 6
SELECT * INTO #TEMP3
FROM (SELECT * FROM #Temp1 UNION ALL SELECT * FROM #temp2) X

UNION ALL adds all the records where as UNION adds only new/distinct records.
Since you are using UNION ALL and using DISTINCT soon after, I think you are looking for UNION
SELECT * INTO #TEMP4 FROM
(
SELECT * FROM #TEMP3
UNION --JUST UNION
SELECT * FROM #TEMP2
) AnotherTemp
Or you can simplify it as
SELECT * INTO #TEMP4 FROM
SELECT DISTINCT *
FROM Final
WHERE MomentId = #MomentId OR RowNum BETWEEN #StartRow AND #EndRow

I'm not familiar with SQL-Server, but you might get my idea
select *, 'A' tid, rownumber() tno from tableA
union all
select *, 'B', rownumber() from tableB
order by tid, tno;
This should get you all records of tableA in their specific order, followed by all records of tableB in their specific order.

Related

Perform a function after union of two tables

I have two tables that I want to union together then perform some math functions on the combined table.
I know how to do the math for each separate table, but throwing in a union table to go off of is out of my league.
Here's the math for one table using column header "UnitsReceived" and "AsnPsUnits"
The other table would have headers: "cUnitsReceived" and "cAsnPsUnits"
select VendName,
1-abs(((cast(sum(UnitsReceived) as decimal(5,0))) - (cast(sum(AsnPsUnits) as decimal(5,0)))) /(cast(sum(AsnPsUnits) as decimal(5,0)))) as ASNpsAcc
from VenTest2
where ID<20
group by VendName
How would I perform this function after the union of two tables?
You'll need to get the unioned tables into some table object before performing your function. This could be done using:
A Common Table Expression
with cte as (
select ID, VALUE from A
union all
select ID, VALUE from B
)
select
*
,myfunction(VALUE) as MyFunctionResult
from
cte
A temp table
select ID, VALUE into #myTempTable from A
insert into #myTempTable select ID, VALUE from B
select
*
,myfunction(VALUE) as MyFunctionResult
from
#myTempTable
A table variable
declare #myTableVariable table (ID int, VALUE decimal)
insert into #myTableVariable
select ID, VALUE from A
union all
select ID, VALUE from B
select
*
,myfunction(VALUE) as MyFunctionResult
from
#myTableVariable
A sub query
select
*
,myfunction(VALUE) as MyFunctionResult
from
(
select ID, VALUE from A
union all
select ID, VALUE from B
) mySubQuery
This will help with the subq being the union
select VendName,
1-abs(((cast(sum(UnitsReceived) as decimal(5,0))) - (cast(sum(AsnPsUnits) as decimal(5,0)))) /(cast(sum(AsnPsUnits) as decimal(5,0)))) as ASNpsAcc
from
(
select ID, UnitsReceived, AsnPsUnits from VenTest2 where ID<20
union
select ID1, UnitsReceived1, AsnPsUnits1 from VenTest1
)a
group by VendName
This is not the way, brothers:
select VendName,
1-abs(((cast(sum(UnitsReceived) as decimal(10,2))) - (cast(sum(AsnPsUnits) as decimal(10,2)))) /(cast(sum(AsnPsUnits) as decimal(10,2)))) as ASNpsAcc
from VenTest2
where ID<10
group by VendName
union
select cVendName,
1-abs(((cast(sum(cUnitsReceived) as decimal(10,2))) - (cast(sum(casnpsunits) as decimal(10,2)))) /(cast(sum(cAsnPsUnits) as decimal(10,2)))) as ASNpsAcc
from CTest
where id <10
group by cvendname

Merge three tables in Select query by rule 3, 2, 1 records from each table

Merge three tables in a Select query by rule 3, 2, 1 records from each table as follows:
TableA: ID, FieldA, FieldB, FieldC,....
TableB: ID, FieldA, FieldB, FieldC,....
TableC: ID, FieldA, FieldB, FieldC,....
ID : auto number in each table
FieldA will be unique in all three tables.
I am looking for a Select query to merge three tables as follows:
TOP three records from TableA sorted by ID
TOP two records from TableB sorted by ID
TOP 1 record from TableC sorted by ID
Repeat this until select all records from all three tables.
If some table has fewer records or does not meet the criteria, ignore that and continue with others.
My attempt:
I did it totally through programming way, like cursors and If conditions inside a SQL Server stored procedure.
It makes delay.
This requires a formula that takes row numbers from each table and transforms it into a series of integers that skips the desired values.
In the query below, I am adding some CTE for the sake of shortening the formula. The real magic is in the UNION. Also, I am adding an additional field for your control. Feel free to get rid of it.
WITH A_Aux as (
SELECT 'A' As FromTable, ROW_NUMBER() OVER (ORDER BY ID) AS RowNum, TableA.*
FROM TableA
), B_Aux AS (
SELECT 'B' As FromTable, ROW_NUMBER() OVER (ORDER BY ID) AS RowNum, TableB.*
FROM TableB
), C_Aux AS (
SELECT 'C' As FromTable, ROW_NUMBER() OVER (Order BY ID) AS RowNum, TableC.*
FROM TableC
)
SELECT *
FROM (
SELECT RowNum+3*FLOOR((RowNum-1)/3) As ColumnForOrder, A_Aux.* FROM A_Aux
UNION ALL
SELECT 3+RowNum+4*FLOOR((RowNum-1)/2), B_Aux.* FROM B_Aux
UNION ALL
SELECT 6*RowNum, C_Aux.* FROM C_Aux
) T
ORDER BY ColumnForOrder
PS: note the pattern Offset + RowNum + (6-N) * Floor((RowNum-1)/N) to group N records together (it of course simplifies a lot for TableC).
PPS: I don't have a SQL server at hand to test it. Let me know if there is a syntax error.
You may try this..
GO
select * into #temp1 from (select * from table1) as t1
select * into #temp2 from (select * from table2) as t2
select * into #temp3 from (select * from table3) as t3
select * into #final from (select col1, col2, col3 from #temp1 where 1=0) as tb
declare #i int
set #i=1
while( (select COUNT(*) from #temp1)>#i)
Begin
;with ct1 as (
select ROW_NUMBER() over (order by id) as Slno, * from #temp1
),ct2 as (
select ROW_NUMBER() over (order by id) as Slno, * from #temp2
),ct3 as (
select ROW_NUMBER() over (order by id) as Slno, * from #temp3
),cfinal as (
select top 3 * from #temp1
union all
select top 2 * from #temp2
union all
select top 1 * from #temp3
)
insert into #final ( col1 , col2, col3 )
select col1, col2, col3 from cfinal
delete from #temp1 where id in (select top 3 ID from #temp1)
delete from #temp2 where id in (select top 2 ID from #temp2)
delete from #temp3 where id in (select top 1 ID from #temp3)
set #i = #i+1
End
Select * from #final
Drop table #temp1
Drop table #temp2
Drop table #temp3
GO
First create temp table for all 3 tables with each insert delete the inserted record and this will result you the desired result, if nothing is missing from my side.
Please see to this if this works.
There is not a lot of information to go with here, but I assume you can use UNION to combine multiple statements.
SELECT * TableA ORDER BY ID DESC OFFSET 3 ROWS
UNION
SELECT * TableB ORDER BY ID DESC OFFSET 2 ROWS
UNION
SELECT * TableC ORDER BY ID DESC OFFSET 1 ROWS
Execute and see if this works.
/AF
From my understanding, I create three temp tables as ta, tb, tc.
select * into #ta from (
select 'A' a
union all
select 'A' a
union all
select 'A' a
union all
select 'A' a
union all
select 'A' a
union all
select 'A' a
union all
select 'A' a
) a
select * into #tb from (
select 'B' b
union all
select 'B'
union all
select 'B'
union all
select 'B'
union all
select 'B'
) b
select * into #tc from (
select 'C' c
union all
select 'C'
union all
select 'C'
union all
select 'C'
union all
select 'C'
) c
If tables match you tables, then the output looks like A,A,A,B,B,C,A,A,A,B,B,C,A,B,C,C,C
T-SQL
declare #TAC int = (select count (*) from #ta) -- Table A Count = 7
declare #TBC int = (select count (*) from #tb) -- Table B Count = 5
declare #TAR int = #TAC % 3 -- Table A Reminder = 1
declare #TBR int = #TBC % 2 -- Table B Reminder = 1
declare #TAQ int = (#TAC - #TAR) / 3 -- Table A Quotient = (7 - 1) / 3 = 2, is will passed on NTILE
-- So we gonna split as two group (111), (222)
declare #TBQ int = (#TBC - #TBR) / 2 -- Table B Quotient = (5 - 1) / 2 = 2, is will passed on NTILE
-- So we gonna split as two group (11), (22)
select * from (
select *, NTILE (#TAQ) over ( order by a) FirstOrder, 1 SecondOrder from (
select top (#TAC - #TAR) * from #ta order by a
) ta -- 6 rows are obtained out of 7.
union all
select *, #TAQ + 1, 1 from (
select top (#TAR) * from #ta order by a desc
) ta -- Remaining one row is obtained. Order by desc is must
-- Here FirstOrder is next value of previous value.
union all
select *, NTILE (#TBQ) over ( order by b), 2 from (
select top (#TBC - #TBR) * from #tb order by b
) tb
union all
select *, #TBQ + 1, 2 from (
select top (#TBR) * from #tb order by b desc
) tb
union all
select *, ROW_NUMBER () over (order by c), 3 from #tc
) abc order by FirstOrder, SecondOrder
Let me explain the T-SQL:
Before that, FYR: NTILE and Row Number
Get the count.
Find the Quotient which will pass to NTILE function.
Order by the NTILE value and static.
Note:
I am using SQL Server 2017.
If T-SQL works fine, then you need to change the column in order by <yourcolumn>.

How to join multiple CTE's into a temp table

I have been given a script to clean up which uses approx 85 temp tables, I have been advised to use Common Table Expressions.
I have 3 CTE's, the first is the result of 7 tables pulled together using Union all. Followed by 2 more CTE's. The script runs up to:
select * from CTE_1
Union all
select * from CTE_2
Union all
select * from CTE_3
I then want to put all these results into a reusable table so I can then add some joins with various case statement logic. How can I put these into a temp table so that I can reference it later.
I'm looking to reduce the amount of temp tables so rather than put each CTE into a temp table I would ideally put multiple CTE's into one temp table. I currently have:
; with [CTE One] as (
select 1 as col
),
[CTE Two] as (
select 2 as col
),
[CTE Three] as (
select 3 as col
)
select * from CTE_1
Union all
select * from CTE_2
Union all
select * from CTE_3
Alternatively..
IF ( OBJECT_ID('tempdb..#temptable') IS NOT NULL )
BEGIN
DROP TABLE #temptable
END
CREATE TABLE #temptable
(
val int
)
;
WITH [CTE One]
AS ( SELECT 1 AS col
),
[CTE Two]
AS ( SELECT 2 AS col
),
[CTE Three]
AS ( SELECT 3 AS col
)
INSERT INTO #temptable (val)
SELECT *
FROM ( SELECT *
FROM CTE_1
UNION ALL
SELECT *
FROM CTE_2
UNION ALL
SELECT *
FROM CTE_3
) T
Can't you just use into?
select *
into #temptable
from CTE_1
Union all
select * from CTE_2
Union all
select * from CTE_3;
I might also be inclined to use a table variable, if the code is structured appropriately.

Run UNION SELECT only if the fist select is not empty

I have the following stored procedure that returns rows for an endless scroll page. How can I check that newTable is empty? If it's empty then I don't want the UNION SELECT to be executed. Note that newTable is a very complex query so I don't want to execute more than once.
#offset INT,
#fetch INT
WITH newTable AS
(
SELECT * FROM table1
ORDER BY id OFFSET #offset ROWS FETCH NEXT #fetch ROWS ONLY
)
Don't run the following if newTable has reached the last row and is empty
SELECT * FROM newTable
UNION
SELECT * FROM table3
UNION
SELECT * FROM table4
ORDER BY id
You can probably do something like this:
DECLARE #offset INT,
#fetch INT;
WITH newTable AS
(
SELECT * FROM table1
ORDER BY id OFFSET #offset ROWS FETCH NEXT #fetch ROWS ONLY
)
SELECT *
FROM newTable
UNION
SELECT *
FROM table3
WHERE EXISTS (SELECT 1 FROM newTable)
UNION
SELECT *
FROM table4
WHERE EXISTS (SELECT 1 FROM newTable)
ORDER BY id
I'm pretty sure it will not execute the cte again. If it does, you can use a temporary table instead of a cte:
SELECT <ColumnsList> INTO #TemporaryTable
FROM....
and then your UNION query.
SELECT *
FROM #TemporaryTable
UNION
SELECT *
FROM table3
WHERE EXISTS (SELECT 1 FROM #TemporaryTable)
UNION
SELECT *
FROM table4
WHERE EXISTS (SELECT 1 FROM #TemporaryTable)
ORDER BY id

SQL Select Condition Question

I have a quick question about a select statement condition.
I have the following table with the following items. What I need to get is the object id that matches both type id's.
TypeId ObjectId
1 10
2 10
1 11
So I need to get both object 10 because it matches type id 1 and 2.
SELECT ObjectId
FROM Table
WHERE TypeId = 1
AND TypeId = 2
Obviously this doesn't work because it won't match both conditions for the same row. How do I perform this query?
Also note that I may pass in 2 or more type id's to narrow down the results.
Self-join:
SELECT t1.ObjectId
FROM Table AS t1
INNER JOIN Table AS t2
ON t1.ObjectId = t2.ObjectId
AND t1.TypeId = 1
AND t2.TypeId = 2
Note sure how you want the behavior to work when passing in values, but that's a start.
I upvoted the answer from #Cade Roux, and that's how I would do it.
But FWIW, here's an alternative solution:
SELECT ObjectId
FROM Table
WHERE TypeId IN (1, 2)
GROUP BY ObjectId
HAVING COUNT(*) = 2;
Assuming uniqueness over TypeId, ObjectId.
Re the comment from #Josh that he may need to search for three or more TypeId values:
The solution using JOIN requires a join per value you're searching for. The solution above using GROUP BY may be easier if you find yourself searching for an increasing number of values.
This code is written with Oracle in mind. It should be general enough for other flavors of SQL
select t1.ObjectId from Table t1
join Table t2 on t2.TypeId = 2 and t1.ObjectId = t2.ObjectId
where t1.TypeId = 1;
To add additional TypeIds, you just have to add another join:
select t1.ObjectId from Table t1
join Table t2 on t2.TypeId = 2 and t1.ObjectId = t2.ObjectId
join Table t3 on t3.TypeId = 3 and t1.ObjectId = t3.ObjectId
join Table t4 on t4.TypeId = 4 and t1.ObjectId = t4.ObjectId
where t1.TypeId = 1;
Important note: as you add more joins, performance will suffer a LOT.
In regards to Bill's answer you can change it to the following to get rid of the need to assume uniqueness:
SELECT ObjectId
FROM (SELECT distinct ObjectId, TypeId from Table)
WHERE TypeId IN (1, 2)
GROUP BY ObjectId
HAVING COUNT(*) = 2;
His way of doing it scales better as the number of types gets larger.
Try this
Sample Input:(Case 1)
declare #t table(Typeid int,ObjectId int)
insert into #t
select 1,10 union all select 2,10 union all
select 1,11
select * from #t
Sample Input:(Case 2)
declare #t table(Typeid int,ObjectId int)
insert into #t
select 1,10 union all select 2,10 union all
select 3,10 union all select 4,10 union all
select 5,10 union all select 6,10 union all
select 1,11 union all select 2,11 union all
select 3,11 union all select 4,11 union all
select 5,11 union all select 1,12 union all
select 2,12 union all select 3,12 union all
select 4,12 union all select 5,12 union all
select 6,12
select * from #t
Sample Input:(Case 3)[Duplicate entries are there]
declare #t table(Typeid int,ObjectId int)
insert into #t
select 1,10 union all select 2,10 union all
select 1,10 union all select 2,10 union all
select 3,10 union all select 4,10 union all
select 5,10 union all select 6,10 union all
select 1,11 union all select 2,11 union all
select 3,11 union all select 4,11 union all
select 5,11 union all select 1,12 union all
select 2,12 union all select 3,12 union all
select 4,12 union all select 5,12 union all
select 6,12 union all select 3,12
For case 1, the output should be 10
For case 2 & 3, the output should be 10 and 12
Query:
select X.ObjectId from
(
select
T.ObjectId
,count(ObjectId) cnt
from(select distinct ObjectId,Typeid from #t)T
where T.Typeid in(select Typeid from #t)
group by T.ObjectId )X
join (select max(Typeid) maxcnt from #t)Y
on X.cnt = Y.maxcnt