finding the missing values in a Sequence - sql-server-2005

Table1 is as follows :
Col1
1
2
3
4
6
7
8
9
10
13
14
As shown above the col1 has the sequence of values but for some reason the user did not insert 5, 11 and so on. How to find out the missing values in a sequence. Here the sequence is 1 to 14 and the missing values are 5,11. Please help me.

As was said in other answers, the best choice is to do a join with a real sequence table. You can create one using a recursive CTE:
DECLARE #MaxNumber INT
SELECT #MaxNumber = MAX(Col1) FROM YourTable;
WITH CTE AS
(
SELECT 1 Col1
UNION ALL
SELECT Col1+1
FROM CTE
WHERE Col1+1 <= #MaxNumber
)
SELECT A.Col1
FROM CTE A
LEFT JOIN YourTable B
ON A.Col1 = B.Col1
WHERE B.Col1 IS NULL
OPTION(MAXRECURSION 0)

This will work for numbers 0 - 2000 for large numbers you just need to cross join the original result set.
with temp as (
select distinct number
from master..spt_Values
where number between 0 and 2000
)
select * from
temp t
left join your_table y on y.col1 = t.number
where y.col1 is null
alternatively using cross join
This will work for billions obviously slower
WITH
L0 AS(SELECT 1 AS c UNION ALL SELECT 1),
L1 AS(SELECT 1 AS c FROM L0 AS A CROSS JOIN L0 AS B),
L2 AS(SELECT 1 AS c FROM L1 AS A CROSS JOIN L1 AS B),
L3 AS(SELECT 1 AS c FROM L2 AS A CROSS JOIN L2 AS B),
L4 AS(SELECT 1 AS c FROM L3 AS A CROSS JOIN L3 AS B),
L5 AS(SELECT 1 AS c FROM L4 AS A CROSS JOIN L4 AS B),
Nums AS(SELECT ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) AS n FROM L5)
select * from
l5 t
left join your_table y on y.col1 = t.n
where y.col1 is null

This seems to pretty much be a duplication of
SQL query to find Missing sequence numbers
There's a suggestions this will work:
SELECT l.id + 1 as start
FROM Table1 as l
LEFT JOIN Table1 as r on l.id + 1 = r.id
WHERE r.id IS NULL
Otherwise you can left join on your table with a sequence table. From the above question, you can look at http://www.projectdmx.com/tsql/tblnumbers.aspx to get some ideas on how to generate a suitable sequence table, and the join will be something like
SELECT #sequence.value
FROM #sequence
LEFT JOIN Table1 ON #sequence.value = Table1.value
WHERE Table1.value IS NULL

Side-note to all recursive CTE suggestions. The recursive CTE increases time linear to the number of rows. Using a tally table or cross-join is much better to use...
This would work:
-- data table
CREATE TABLE #data (
value INT
)
INSERT #data VALUES (1)
INSERT #data VALUES (2)
INSERT #data VALUES (3)
INSERT #data VALUES (4)
INSERT #data VALUES (6)
INSERT #data VALUES (7)
INSERT #data VALUES (8)
INSERT #data VALUES (9)
INSERT #data VALUES (10)
INSERT #data VALUES (13)
INSERT #data VALUES (14)
-- normally i have a tally table already for stuff like this but I'll
-- create one temporary here.
CREATE TABLE #tmp_tally (
n INT
)
DECLARE #n INT
SET #n = 1
WHILE #n < 14
BEGIN
INSERT #tmp_tally VALUES (#n)
SET #n = #n + 1
END
SELECT
T.n,
CASE WHEN #data.value IS NULL THEN 'Missing' ELSE 'Not Missing' END
FROM
#tmp_tally T
LEFT JOIN #data ON
T.n = #data.value
WHERE
T.n <= (SELECT MAX(value) FROM #data) -- max of what you want to check against which is 14 in your example
DROP TABLE #data
DROP TABLE #tmp_tally

Try this:
declare #min int
declare #max int
select #min = min(field_ID), #max = max(field_ID) from [Table]
create table #tmp (Field_No int)
while #min <= #max
begin
if not exists (select * from [Table] where field_ID = #min)
insert into #tmp (seq_field) values (#min)
set #min = #min + 1
end
select * from #tmp
drop table #tmp
With the above script you will get missing values in "ID" column from #tmp table.
Hope this will help you!!

I would do a subquery in the same table, to see if another number exist for the current number-1, and if there is not one, you know that a number was skipped. You can do the +1 of this as well.
select
nt.numb,
CASE
(select COUNT(*) from table where numb=nt.numb-1)=0 THEN 'skipped' ELSE 'not skipped'
from
numbertable nt

Related

Crash gaps in a SQL string [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 4 years ago.
Improve this question
I have a select today that returns me the following result:
I make another select that returns a value, 5 for example. I would need to know which numbers are not between the number 1 to the number 5. In my case I would need the following result 2, 4, 5.
I would need help developing this logic.
I have created a sample using NOT EXISTS(), I believe it can help you :
--DROP TABLE #Temp
--GO
CREATE TABLE #Temp
(
NR_VOLUME INT
)
INSERT INTO #Temp
VALUES (1),(3)
--Create Temp table for number List
--DROP TABLE #NumberList
--GO
CREATE TABLE #NumberList
(
Seq INT
)
DECLARE #NumberCheckFrom INT=1
DECLARE #NumberCheckTo INT=100
WHILE #NumberCheckTo>=#NumberCheckFrom
BEGIN
INSERT INTO #NumberList
SELECT #NumberCheckFrom
SET #NumberCheckFrom +=1
END
DECLARE #NumberFrom INT=0
DECLARE #NumberTo INT=5
SELECT *
FROM #NumberList NL
WHERE Seq>=#NumberFrom
AND Seq<=#NumberTo
AND NOT EXISTS (SELECT 1
FROM #Temp T
WHERE NL.Seq = T.NR_VOLUME
)
Build a cte with the values you are looking for. Then get the values that dont exist in your table
with cte (rn) as (select 1 as rn
union all
select rn + 1
from cte
where rn + 1 <= 5)
select cte.rn
from cte
where not exists (select 1 from actualtable where nr_volume = cte.rn)
Your question wasn't clear, but it seems you wanted numbers that weren't present in the range returned by two queries, but fell within that range. For that, you can use a tally table and a few methods to limit the results. Note, you included 5 in your expected results, but this contradicts your question since you didn't include 1 int he expected results.
declare #table1 table (i int)
insert into #table1
values (1),(3)
declare #table2 table (i int)
insert into #table2
values (5)
declare #max int = (select top 1 i from (select max(i) i from #table1 union select max(i) from #table2) x order by i desc)
declare #min int = (select top 1 i from (select min(i) i from #table1 union select min(i) from #table2) x order by i asc)
;WITH
E1(N) AS (select 1 from (values (1),(1),(1),(1),(1),(1),(1),(1),(1),(1))dt(n)),
E2(N) AS (SELECT 1 FROM E1 a, E1 b), --10E+2 or 100 rows
E4(N) AS (SELECT 1 FROM E2 a, E2 b), --10E+4 or 10,000 rows max
cteTally(N) AS
(
SELECT ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) FROM E4
)
select N from cteTally
where
N not in (select * from #table1)
and N not in (select * from #table2)
and N > #min
and N < #max
GO

SQL multiplying rows in select

I would like to select some rows multiple-times, depending on the column's value.
Source table
Article | Count
===============
A | 1
B | 4
C | 2
Wanted result
Article
===============
A
B
B
B
B
C
C
Any hints or samples, please?
You could use:
SELECT m.Article
FROM mytable m
CROSS APPLY (VALUES (1),(2),(3),(4),(5),(6),(7),(8),(9),(10)) AS s(n)
WHERE s.n <= m.[Count];
LiveDemo
Note: CROSS APLLY with any tally table. Here values up to 10.
Related: What is the best way to create and populate a numbers table?
You could also use a recursive CTE which works with numbers > 10 (here up to 1000):
With NumberSequence( Number ) as
(
Select 0 as Number
union all
Select Number + 1
from NumberSequence
where Number BETWEEN 0 AND 1000
)
SELECT Article
FROM ArticleCounts
CROSS APPLY NumberSequence
WHERE Number BETWEEN 1 AND [Count]
ORDER BY Article
Option (MaxRecursion 0)
Demo
A number-table will certainly be the best option.
http://sqlperformance.com/2013/01/t-sql-queries/generate-a-set-2
Please check following SQL script
Before executing the SELECT statement, note that I used a user function which is used to simulate a numbers table
You can find the sql codes of numbers table in SQL Server at referred tutorial
----create table myTempTbl (Article varchar(10), Count int)
--insert into myTempTbl select 'A',1
--insert into myTempTbl select 'B',4
--insert into myTempTbl select 'C',2
select t.*
from myTempTbl t
cross apply dbo.NumbersTable(1,100,1) n
where n.i <= t.Count
order by t.Article
one more CTE
with cte_t as (
select c as c, 1 as i
from mytable
group by c
union all
select t.c, ctet.i + 1
from mytable t
join cte_t ctet
on ctet.c = t.c
and ctet.i < t.i
)
select cte_t.c
from cte_t
order by cte_t.c
Can obtain the output using simple WHILE LOOP
DECLARE #table TABLE
(ID int ,Article varchar(5),[Count] int)
INSERT INTO #table
(ID,Article,Count)
VALUES
(1,'A',1),(2,'B',4),(3,'C',2)
DECLARE #temp TABLE
(Article varchar(5))
DECLARE #Cnt1 INT
DECLARE #Cnt2 INT
DECLARE #Check INT
DECLARE #max INT
SET #max =0
SET #Cnt1 = (SELECT Count(Article) FROM #table)
WHILE (#max < #Cnt1)
BEGIN
SET #max = #max +1
SET #Cnt2 = (SELECT [Count] FROM #table WHERE ID =#max)
SET #Check =(SELECT [Count] FROM #table WHERE ID =#max)
WHILE (#Cnt2 > 0)
BEGIN
INSERT INTO #temp
SELECT Article FROM #table WHERE [Count] =#Check
SET #Cnt2 = #Cnt2 -1
END
END
SELECT * FROM #temp

Fastest way to insert 100000 records into SQL Server

I am using the following script to insert 100,000 records into a table. Basically int from 500,001 to 600,000 are inserted. I am casting the integer into a string and inserting coz thats how i want it in the table (an integer in the form of string). I am using a merge to check if the record already exists or not.
DECLARE #first AS INT
SET #first = 500001
DECLARE #step AS INT
SET #step = 1
DECLARE #last AS INT
SET #last = 600000
BEGIN TRANSACTION
WHILE(#first <= #last)
BEGIN
MERGE dbo.Identifiers As target
USING (SELECT CAST(#first as varchar(10)) AS Identifier) AS source
ON (source.Identifier = target.Identifier)
WHEN NOT MATCHED THEN
INSERT (Identifier)
VALUES (source.Identifier);
SET #first += #step
END
COMMIT TRANSACTION
Its taking more than 2 minutes to load. I am doing something terribly wrong but unable to trace out where.
Note: The table has unique non-clustered index on Identifier Column.
I am wondering how much your procedural looping and the MERGE (instead of a simple INSERT) contributes to bad performance. I would opt for a strictly set-based solution like this:
INSERT INTO dbo.Identifiers (Identifier)
SELECT n FROM dbo.GetNums(500001, 600000)
WHERE n NOT IN (SELECT Identifier FROM dbo.Identifiers);
Now, this relies on a user-defined table-valued function dbo.GetNums that returns a table containing all numbers between 500,001 and 600,000 in a column called n. How do you write that function? You need to generate a range of numbers on the fly inside it.
The following implementation is taken from the book "Microsoft SQL Server 2012 High-Performance T-SQL Using Window Functions" by Itzik Ben-Gak.
CREATE FUNCTION dbo.GetNums(#low AS BIGINT, #high AS BIGINT) RETURNS TABLE
AS
RETURN
WITH L0 AS (SELECT c FROM (VALUES(1),(1)) AS D(c)),
L1 AS (SELECT 1 AS c FROM L0 AS A CROSS JOIN L0 AS B),
L2 AS (SELECT 1 AS c FROM L1 AS A CROSS JOIN L1 AS B),
L3 AS (SELECT 1 AS c FROM L2 AS A CROSS JOIN L2 AS B),
L4 AS (SELECT 1 AS c FROM L3 AS A CROSS JOIN L3 AS B),
L5 AS (SELECT 1 AS c FROM L4 AS A CROSS JOIN L4 AS B),
Nums AS (SELECT ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS rownum FROM L5)
SELECT #low + rownum - 1 AS n
FROM Nums
ORDER BY rownum
OFFSET 0 ROWS FETCH FIRST #high - #low + 1 ROWS ONLY;
(Since this comes from a book on SQL Server 2012, it might not work on SQL Server 2008 out-of-the-box, but it should be possible to adapt.)
Try this one. It uses a tally table. Reference: http://www.sqlservercentral.com/articles/T-SQL/62867/
create table #temp_table(
N int
)
declare #first as int
set #first = 500001
declare #step as int
set #step = 1
declare #last as int
set #last = 600000
with
e1 as(select 1 as N union all select 1), --2 rows
e2 as(select 1 as N from e1 as a, e1 as b), --4 rows
e3 as(select 1 as N from e2 as a, e2 as b), --16 rows
e4 as(select 1 as N from e3 as a, e3 as b), --256 rows
e5 as(select 1 as N from e4 as a, e4 as b), --65,356 rows
e6 as(select 1 as N from e5 as a, e1 as b), -- 131,072 rows
tally as (select 500000 + (row_number() over(order by N) * #step) as N from e6) -- change 500000 with desired start
insert into #temp_table
select cast(N as varchar(10))
from tally t
where
N >= #first
and N <=#last
and not exists(
select 1 from #temp_table where N = t.N
)
drop table #temp_table
Vinoth, Something given below could also help you.
Declare #tab table (id int identity(1,1),num int)
Insert into #tab (num) Values (1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
Declare #start as int
set #start = 500000
Insert into dbo.Identifiers (Identifier)
Select #start + ((E.id-1)*10000) +((D.id-1)*1000) +((C.id-1)*100) + ((B.id-1) * 10) + A.id
from #tab A,#tab B,#tab C,#tab D,#tab E
Order by #start + ((E.id-1)*10000) +((D.id-1)*1000) +((C.id-1)*100) + ((B.id-1) * 10) + A.id
In my DB, dbo.Identifiers is a table without any index. It took only 230 ms for the insertion.
Create index on the Identifier column and then try the above insert

Select non-existing rows

Let say I have a table:
ColumnA ColumnB
---------------------------------
1 10.75
4 1234.30
6 2000.99
How can I write a SELECT query that will result in the following:
ColumnA ColumnB
---------------------------------
1 10.75
2 0.00
3 0.00
4 1234.30
5 0.00
6 2000.99
You can use a CTE to create a list of numbers from 1 to the maximum value in your table:
; with numbers as
(
select max(ColumnA) as nr
from YourTable
union all
select nr - 1
from numbers
where nr > 1
)
select nr.nr as ColumnA
, yt.ColumnB
from numbers nr
left join
YourTable yt
on nr.nr = yt.ColumnA
order by
nr.nr
option (maxrecursion 0)
See it working at SQL Fiddle.
Please try:
declare #min int, #max int
select #min=MIN(ColumnA), #max=MAX(ColumnA) from tbl
select
distinct number ColumnA,
isnull(b.ColumnB, 0) ColumnB
from
master.dbo.spt_values a left join tbl b on a.number=b.ColumnA
where number between #min and #max
Create a TallyTable (or NumbersTable) - see this question: What is the best way to create and populate a numbers table?
With that table create an insert statement:
INSERT INTO YourTable (ColumnA, ColumnB)
SELECT Number FROM NumberTable
WHERE
NOT EXISTS (SELECT 1 FROM YourTable WHERE NumberTable.Number = YourTable.ColumnA)
-- Adjust this value or calculate it with a query to the maximum of the source table
AND NumberTable.Number < 230130
DECLARE #t TABLE (ID INT,Val DECIMAL(10,2))
INSERT INTO #t (ID,Val) VALUES (1,10.75)
INSERT INTO #t (ID,Val) VALUES (4,6.75)
INSERT INTO #t (ID,Val) VALUES (7,4.75)
declare #MinNo int
declare #MaxNo int
declare #IncrementStep int
set #MinNo = 1
set #MaxNo = 10
set #IncrementStep = 1
;with C as
(
select #MinNo as Num
union all
select Num + #IncrementStep
from C
where Num < #MaxNo
)
select Num,
CASE WHEN Val IS NOT NULL THEN Val ELSE 0.00 END AS NUMBER
from C
LEFT JOIN #t t
ON t.ID = c.Num
You could use a number-table or following trick to generate a sequence which you can LEFT OUTER JOIN with your table. I assume you want to determine the boundaries dynamically:
WITH Seq AS
(
SELECT TOP ((SELECT Max(ColumnA)FROM Table1) - (SELECT Min(ColumnA) FROM Table1) + 1)
Num = (SELECT Min(ColumnA) FROM Table1)+ Row_number() OVER (ORDER BY [object_id]) -1
FROM sys.all_objects)
SELECT ColumnA = Seq.Num,
ColumnB = COALESCE(t.ColumnB ,0.00)
FROM Seq
LEFT OUTER JOIN Table1 t
ON Seq.Num = t.ColumnA
Demo with your sample.
Worth reading: http://www.sqlperformance.com/2013/01/t-sql-queries/generate-a-set-1
I have my collect of table functions like these.
create function dbo.GetNumbers(#Start int, #End int)
returns #Items table
(
Item int
)
as
begin
while (#Start <= #End)
begin
insert into #Items
values (#Start)
set #Start = #Start + 1
end
return
end
Then I can use it to left join to my data table and every value will be there.
declare #min int, #max int
set #min = 10
set #max = 20
select gn.Item
from dbo.GetNumbers(#min, #max) gn
I have similar table functions for date ranges, times, timezones, etc.

How to find missing id in the table

I have column looks like below
SID101
SID102
SID103
SID105
SID107
In the above criteria i need to find missed SID numbers. SID104 and SID 106 are missed while ordering.
How can i find the missed id numbers.Could any one help me finding it.
Thanks in advance.
If your table contains gaps with length more than 1 item, you can use this query:
declare #t table(s varchar(20))
insert #t values ('SID101'),('SID102'),('SID103'),('SID105'),('SID108');
with cte as
(
select substring(t.s, 4, len(t.s)) [i]
from #t t
)
select 'SID' + cast(m.number as varchar(20))
from master..spt_values m
left join cte c on c.i = m.number
where [Type] = 'P'
and m.number >= (select min(i) from cte)
and m.number <= (select max(i) from cte)
and c.i is null
Output:
-----------------------
SID104
SID106
SID107
Something like this should work:
DECLARE #i INT;
SET #i = 100;
CREATE TABLE #idsToCheck (checkId varchar(100));
WHILE (#i < 200)
BEGIN
INSERT INTO #idsToCheck VALUES ('SID' + CONVERT(varchar(100), #i));
SET #i = #i + 1;
END
SELECT * FROM #idsToCheck itc
LEFT OUTER JOIN MainTable mt ON itc.checkId = mt.realId
WHERE mt.realId = NULL
DROP TABLE #idsToCheck
... where MainTable is your table containing the SID101, SID102, etc. column values, and MainTable.realId is the column containing those IDs. Modify the #i initial value and number in the while loop condition based on which SIDs you want to check from/to.
It's difficult. With
SELECT COUNT(*),MAX(CAST(REPLACE(y.name,'SID','') AS INT)) AS col_max FROM
sys.objects x INNER JOIN sys.columns y ON x.object_id=y.object_id
WHERE x.name='<TABLE_NAME>'
you should know, how much columns are missing (i.e. COUNT(*) is 5 and col_max is 107)
When you have a table, which contains only one column with all possible IDs from 1 to max (i.e. 100,101,102,103,104,...,132) then you could do
SELECT * FROM (
SELECT CAST(REPLACE(y.name,'SID','') AS INT) AS col_id FROM
sys.objects x INNER JOIN sys.columns y ON x.object_id=y.object_id
WHERE x.name='<TABLE_NAME>'
) a
RIGHT JOIN <TABLE_IDS> b ON a.col_id=b.id
WHERE a.col_id IS NULL AND b.id<=(
SELECT MAX(CAST(REPLACE(y.name,'SID','') AS INT)) AS col_max FROM
sys.objects x INNER JOIN sys.columns y ON x.object_id=y.object_id
WHERE x.name='<TABLE_NAME>'
)
EDIT: sorry, I've seen just now, that these values aren't column names, but values. My solution will find missing column names
 Declare #St int
declare #end int
set #st = CAST( (select RIGHT( max(data),4) from orderno)as int)
set #end = CAST( (select RIGHT( min(data),4) from orderno)as int)
create table #temp(data int)
while(#St <= #end )
begin
insert into #temp values(#St)
set #St = #St +1
end
select * from orderno
select * from #temp
select data from #temp where data not in (select cast(RIGHT(data,4))
declare #t table(s varchar(20))
insert #t values ('SID101'),('SID102'),('SID103'),('SID105'),('SID107');
with cte as
(
select substring(t.s, 4, len(t.s)) [i]
from #t t
)
select 'SID' + cast(t1.i + 1 as varchar(20))
from cte t1
join cte t2 on t2.i > t1.i
and not exists(
select 1
from cte c3
where c3.i > t1.i and c3.i < t2.i
)
where t2.i <> t1.i + 1
Output:
-----------------------
SID104
SID106