sql combination - sql

I have a sql query which select the best combination of hours with the heighest weight, It uses a recursive CTE to do find all the combinations of hours as shown below
Declare #EmpClasses table(Id int identity(1,1),ClassNum int,ClassWeight int,ClassHours int)
Declare #HoursReq int;
SET #HoursReq = 20;
INSERT INTO #EmpClasses VALUES(1001,10,10),(1002,9,5),(1003,8,4),(1004,7,3),(1005,6,2),(1006,5,2),(1007,4,1);
--INSERT INTO #EmpClasses VALUES(1001,2,2),(1002,2,2),(1003,2,2),(1004,2,2),(1005,2,2),(1006,2,2),(1007,2,2),(1008,2,2),(1009,2,2),(1010,2,2);
--INSERT INTO #EmpClasses VALUES(1011,2,2),(1012,2,2),(1013,2,2),(1014,2,2),(1015,2,2),(1016,2,2),(1017,2,2),(1018,2,2),(1019,2,2),(1020,2,2);
--INSERT INTO #EmpClasses VALUES(1021,2,2),(1022,2,2),(1023,2,2),(1024,2,2),(1025,2,2),(1026,2,2),(1027,2,2),(1028,2,2),(1029,2,2),(1030,2,2);
WITH cte (Id,comIds,Total,TotalWeight)
AS
(
SELECT Id
,Cast(ClassNum as varchar(max)) + ','
,ClassHours
,ClassWeight
FROM #EmpClasses
UNION ALL
SELECT ec.Id
,cte.comIds + Cast(ec.ClassNum as varchar(max)) + ','
,cte.Total + ec.ClassHours
,cte.TotalWeight + ec.ClassWeight
FROM #EmpClasses AS ec JOIN cte ON ec.Id < cte.Id
)
SELECT top(1)comids,Total,TotalWeight
FROM cte
Where Total = #HoursReq
order by TotalWeight desc
However the problems lies, when the number of rows increases to iterate. It takes a very long time.
Note:- Please uncomment above lines to check for the issue
Any help regarding this will be greatly appreciated


The number of rows generated in cte is 2^n-1 where n is the number of records in the #EmpClasses table.
So, if you use just 7 records, you generate 127 records.
However if you use 37 records, as in your full example, it needs to generate 137.438.953.471 rows. That takes some time, even if you have a fast server.
You can verify if you change the selection at the bottom of the cte, with a simple
select * from cte

Related

Generate a list with string prefix in SQL with fixed length

I just want to generate a list like this
XY0001
XY0002
XY0003
The prefix is same for all rows. Need fixed length (6 in this example)
Looking for an easy way to produce such list to put it into temp table.
MS SQL
for a very small number this would do:
DECLARE #TempList TABLE (Name VARCHAR(100));
insert into #TempList Values ('XY00001')
insert into #TempList Values ('XY00002')
insert into #TempList Values ('XY00003')
insert into #TempList Values ('XY00004')
select * from #TempList

You can use an ad-hoc tally table
If 2012+
DECLARE #TempList TABLE (Name VARCHAR(100));
Select Name = 'XY'+format(N,'0000')
From (Select Top 9999 N=Row_Number() Over (Order By (Select NULL)) From master..spt_values N1,master..spt_values N2) A
Order by N
Returns
Name
XY0001
XY0002
XY0003
XY0004
...
XY9997
XY9998
XY9999
If not
DECLARE #TempList TABLE (Name VARCHAR(100));
Select Name = 'XY'+right('00000'+cast(N as varchar(25)),4)
From (Select Top 9999 N=Row_Number() Over (Order By (Select NULL)) From master..spt_values N1,master..spt_values N2) A
Order by N

I like to use recursive CTE's for this.
declare #max_number int = 1000;
with num as (
select 1 as n
union
select n + 1
from num
where n < #max_number
)
select 'XY' + (cast n as char(4))
from num;
The recursive CTE gives you the numbers and the cast does the left-padding with 0's to ensure you get 0001 instead of 1.
This approach will support a variable number of outputs. Though as you alluded to in your question, this is overkill if you only want a few.
(You'll need to test this out for boundary cases. I haven't tested this exact code sample.)
There is likely a limit to how far this scales because it uses recursion.

SQL Server recent rows

I'm sure this is easy but I have googled a lot and searched.
Ok, I have a table WITHOUT dates etc with 100000000000000000 records.
I want to see the latest entries, i.e.
Select top 200 *
from table
BUT I want to see the latest entries. Is there a rowidentifier that I could use in a table?
ie
select top 200 *
from table
order by rowidentifer Desc
Thanks

Is there a row.identifier that i could use in a table ie set top 200 * from table order by row.identifer Desc
As already stated in the comment's, there is not. The best way is having an identity, timestamp or some other form of identifying the record. Here is an alternative way using EXCEPT to get what you need, but the execution plan isn't the best... Play around with it and change as needed.
--testing purposes...
DECLARE #tbl TABLE(FirstName VARCHAR(50))
DECLARE #count INT = 0
WHILE (#count <= 12000)
BEGIN
INSERT INTO #tbl(FirstName)
SELECT
'RuPaul ' + CAST(#count AS VARCHAR(5))
SET #count += 1
END
--adjust how many records you would like, example is 200
SELECT *
FROM #tbl
EXCEPT(SELECT TOP (SELECT COUNT(*) - 200 FROM #tbl) * FROM #tbl)
--faster than above
DECLARE #tblCount AS INT = (SELECT COUNT(*) FROM #tbl) - 200
SELECT *
FROM #tbl
EXCEPT(SELECT TOP (#tblCount) * FROM #tbl)
On another note, you could create another Table Variable that has an ID and other columns, then you could insert the records you would need. Then you can perform other operations against the table, for example OrderBy etc...
What you could do
ALTER TABLE TABLENAME
ADD ID INT IDENTITY
This will add another column to the table "ID" and automatically give it an ID. Then you have an identifier you can use...

Nope, in short, there is none, if you don`t have a column dedicated as one (ie. an IDENTITY, or a SEQUENCE, or something similar). If you did, then you could get an ordered result back.

Loop through sql result set and remove [n] duplicates

I've got a SQL Server db with quite a few dupes in it. Removing the dupes manually is just not going to be fun, so I was wondering if there is any sort of sql programming or scripting I can do to automate it.
Below is my query that returns the ID and the Code of the duplicates.
select a.ID, a.Code
from Table1 a
inner join (
SELECT Code
FROM Table1 GROUP BY Code HAVING COUNT(Code)>1)
x on x.Code= a.Code
I'll get a return like this, for example:
5163 51727
5164 51727
5165 51727
5166 51728
5167 51728
5168 51728
This snippet shows three returns for each ID/Code (so a primary "good" record and two dupes). However this isnt always the case. There can be up to [n] dupes, although 2-3 seems to be the norm.
I just want to somehow loop through this result set and delete everything but one record. THE RECORDS TO DELETE ARE ARBITRARY, as any of them can be "kept".

You can use row_number to drive your delete.
ie
CREATE TABLE #table1
(id INT,
code int
);
WITH cte AS
(select a.ID, a.Code, ROW_NUMBER() OVER(PARTITION by COdE ORDER BY ID) AS rn
from #Table1 a
)
DELETE x
FROM #table1 x
JOIN cte ON x.id = cte.id
WHERE cte.rn > 1
But...
If you are going to be doing a lot of deletes from a very large table you might be better off to select out the rows you need into a temp table & then truncate your table and re-insert the rows you need.
Keeps the Transaction log from getting hammered, your CI getting Fragged and should be quicker too!

It is actually very simple:
DELETE FROM Table1
WHERE ID NOT IN
(SELECT MAX(ID)
FROM Table1
GROUP BY CODE)

Self join solution with a performance test VS cte.
create table codes(
id int IDENTITY(1,1) NOT NULL,
code int null,
CONSTRAINT [PK_codes_id] PRIMARY KEY CLUSTERED
(
id ASC
))
declare #counter int, #code int
set #counter = 1
set #code = 1
while (#counter <= 1000000)
begin
print ABS(Checksum(NewID()) % 1000)
insert into codes(code) select ABS(Checksum(NewID()) % 1000)
set #counter = #counter + 1
end
GO
set statistics time on;
delete a
from codes a left join(
select MIN(id) as id from codes
group by code) b
on a.id = b.id
where b.id is null
set statistics time off;
--set statistics time on;
-- WITH cte AS
-- (select a.id, a.code, ROW_NUMBER() OVER(PARTITION by code ORDER BY id) AS rn
-- from codes a
-- )
-- delete x
-- FROM codes x
-- JOIN cte ON x.id = cte.id
-- WHERE cte.rn > 1
--set statistics time off;
Performance test results:
With Join:
SQL Server Execution Times:
CPU time = 3198 ms, elapsed time = 3200 ms.
(999000 row(s) affected)
With CTE:
SQL Server Execution Times:
CPU time = 4197 ms, elapsed time = 4229 ms.
(999000 row(s) affected)

It's basically done like this:
WITH CTE_Dup AS
(
SELECT*,
ROW_NUMBER()OVER (PARTITIONBY SalesOrderno, ItemNo ORDER BY SalesOrderno, ItemNo)
AS ROW_NO
from dbo.SalesOrderDetails
)
DELETEFROM CTE_Dup WHERE ROW_NO > 1;
NOTICE: MUST INCLUDE ALL FIELDS!!
Here is another example:
CREATE TABLE #Table (C1 INT,C2 VARCHAR(10))
INSERT INTO #Table VALUES (1,'SQL Server')
INSERT INTO #Table VALUES (1,'SQL Server')
INSERT INTO #Table VALUES (2,'Oracle')
SELECT * FROM #Table
;WITH Delete_Duplicate_Row_cte
AS (SELECT ROW_NUMBER()OVER(PARTITION BY C1, C2 ORDER BY C1,C2) ROW_NUM,*
FROM #Table )
DELETE FROM Delete_Duplicate_Row_cte WHERE ROW_NUM > 1
SELECT * FROM #Table

SQL: how to get random number of rows from one table for each row in another

I have two tables where the data is not related
For each row in table A i want e.g. 3 random rows in table B
This is fairly easy using a cursor, but it is awfully slow
So how can i express this in single statement to avoid RBAR ?

To get a random number between 0 and (N-1), you can use.
abs(checksum(newid())) % N
Which means to get positive values 1-N, you use
1 + abs(checksum(newid())) % N
Note: RAND() doesn't work - it is evaluated once per query batch and you get stuck with the same value for all rows of tableA.
The query:
SELECT *
FROM tableA A
JOIN (select *, rn=row_number() over (order by newid())
from tableB) B ON B.rn <= 1 + abs(checksum(newid())) % 9
(assuming you wanted up to 9 random rows of B per A)

assuming tableB has integer surrogate key, try
Declare #maxRecs integer = 11 -- Maximum number of b records per a record
Select a.*, b.*
From tableA a Join tableB b
On b.PKColumn % (floor(Rand() * #maxRecs)) = 0

If you have a fixed number that you know in advance (such as 3), then:
select a.*, b.*
from a cross join
(select top 3 * from b) b
If you want a random number of rows from "b" for each row in "a", the problem is a bit harder in SQL Server.

Heres an example of how this could be done, code is self contained, copy and press F5 ;)
-- create two tables we can join
DECLARE #datatable TABLE(ID INT)
DECLARE #randomtable TABLE(ID INT)
-- add some dummy data
DECLARE #i INT = 1
WHILE(#i < 3) BEGIN
INSERT INTO #datatable (ID) VALUES (#i)
SET #i = #i + 1
END
SET #i = 1
WHILE(#i < 100) BEGIN
INSERT INTO #randomtable (ID) VALUES (#i)
SET #i = #i + 1
END
--The key here being the ORDER BY newid() which makes sure that
--the TOP 3 is different every time
SELECT
d.ID AS DataID
,rtable.ID RandomRow
FROM #datatable d
LEFT JOIN (SELECT TOP 3 * FROM #randomtable ORDER BY newid()) as rtable ON 1 = 1
Heres an example of the output

Selecting rows that don't exist physically in the database

I've totally rewritten my question because the simplicity of the previous one people were taking too literally.
The aim:
INSERT INTO X
SELECT TOP 23452345 NEWID()
This query should insert 23452345 GUIDs to the "X" table. actually 23452345 means just any possible number that is entered by user and stored in database.
So the problem is that inserting rows to a database by using
INSERT INTO ... SELECT ...
statement requires you to already have the required amount of rows inserted to database.
Naturally you can emulate the existence of rows by using temporary data and cross joining it but this (in my stupid opinion) creates more results than needed and in some extreme situations might fail due to many unpredicted reasons. I need to be sure that if user entered extremely huge number like 2^32 or even bigger the system will work and behave normally without any possible side effects like extreme memory/time consumption etc...

In all fairness I derived the idea from this site.
;WITH cte AS
(
SELECT 1 x
UNION ALL
SELECT x + 1
FROM cte
WHERE x < 100
)
SELECT NEWID()
FROM cte
EDIT:
The general method we're seeing is to select from a table that has the desired number of rows. It's hackish, but you can create a table, insert the desired number of records, and select from it.
create table #num
(
num int
)
declare #i int
set #i = 1
while (#i <= 77777)
begin
insert into #num values (#i)
set #i = #i + 1
end
select NEWID() from #num
drop table #num

Of course creating a Number table is the best approach and will come in handy. You should definitely have one at your disposal. If you need something as a one-off just join to a known table. I usually use a system table such as spt_values:
declare #result table (id uniqueidentifier)
declare #sDate datetime
set #sDate = getdate();
;with num (n)
as ( select top(777777) row_number() over(order by t1.number) as N
from master..spt_values t1
cross join master..spt_values t2
)
insert into #result(id)
select newid()
from num;
select datediff(ms, #sDate, getdate()) [elasped]

I'd create an integers table and use it. This type of table comes in handy many situations.
CREATE TABLE dbo.Integers
(
i INT IDENTITY(1,1) PRIMARY KEY CLUSTERED
)
WHILE COALESCE(SCOPE_IDENTITY(), 0) <= 100000 /* or some other large value */
BEGIN
INSERT dbo.Integers DEFAULT VALUES
END
Then all you need to do it:
SELECT NEWID()
FROM Integers
WHERE i <= 77777

Try this:
with
L0 as (select 1 as C union all select 1) --2 rows
,L1 as (select 1 as C from L0 as A, L0 as B) --4 rows
,L2 as (select 1 as C from L1 as A, L1 as B) --16 rows
,L3 as (select 1 as C from L2 as A, L2 as B) --256 rows
select top 100 newid() from L3

SELECT TOP 100 NEWID() from sys.all_columns
Or any other datasource that has a large number of records. You can build your own table for 'counting' functionality as such, you can use it in lieu of while loops.
Tally tables: http://www.sqlservercentral.com/articles/T-SQL/62867