How to find a number that isn't there? - sql

Question

We have a custom number field on a training record, the number is recorded sequentially but there are gaps. How do I find those gaps? Consider this pseudocode
SELECT MIN(X)
FROM DUAL
WHERE X BETWEEN 1 AND 999999
AND X NOT IN (SELECT AG_TRNID
FROM PS_TRAINING
WHERE AG_TRNID = X)
This doesn't work, "X" is unknown.
Thanks!
Bruce

Answer 1

This answer assumes that you're using Oracle.
The way to solve this is to create a resultset that contains all of the numbers in your range, then join to that. The way to do this is to use a recursive query:
SELECT LEVEL AS x
FROM DUAL
CONNECT BY LEVEL <= 999999
CONNECT BY is Oracle-specific syntax that tells the query to run recursively as long as the predicate is true. level is a pseudo-column that only exists in queries that use CONNECT BY that indicates the level of recursion. The end result is that this query will run the query against dual 999,999 times, each time being a level deeper in the recursion.
Given this method of generating numbers, plugging it into the query you tried earlier is pretty trivial:
SELECT MIN (x)
FROM (SELECT LEVEL AS x
FROM DUAL
CONNECT BY LEVEL <= 999999)
WHERE x NOT IN (SELECT ag_trnid
FROM ps_training
WHERE ag_trnid = x)

Answer 2

Two quick examples. The first is your classic Gaps-and-Islands, and the second will use an ad-hoc tally table to identify the missings elements via a LEFT JOIN.
The following was created in SQL Server, but if your database supports the window functions, it should be a small task to adapt.
Gaps and Islands
Declare #YourTable table (X int)
Insert Into #YourTable values
(1),(2),(3),(5),(6),(10)
Select X1 = min(X)
,X2 = max(X)
From (
Select *
,Grp = X - Row_Number() over (Order by X)
From #YourTable
) A
Group By Grp
Returns
X1 X2
1 3
5 6
10 10
Ad-hoc Tally Table
Declare #YourTable table (X int)
Insert Into #YourTable values
(1),(2),(3),(5),(6),(10)
;with cte0(N) As (Select 1 From (Values(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)) N(N)),
cteN(N) As (Select Row_Number() over (Order By (Select NULL)) From cte0 N1, cte0 N2, cte0 N3, cte0 N4, cte0 N5, cte0 N6) -- 1 Million
Select N
From cteN
Left Join #YourTable on X=N
Where X is Null
and N<=(Select max(X) from #YourTable)
Order By N
Returns
N
4
7
8
9

Answer 3

Create all numbers from min(ag_trnid) to max(ag_trnid). From these remove the existing numbers:
with nums(num, maxnum) as
(
select min(ag_trnid) as num, max(ag_trnid) as maxnum from ps_training
union all
select num + 1, maxnum from nums where num < maxnum
)
select num from nums
minus
select ag_trnid from ps_training;
Start with 1 (or 0 for that matter) instead of min(ag_trnid), if you consider numbers before the minimum ag_trnid gaps, too.

Answer 4

I think there's no other way besides create a table's id to find the gaps
declare #idMin bigint
declare #idMax bigint
set #idMin = (select min(AG_TRNID) from PS_TRAINING)
set #idMax = (select max(AG_TRNID) from PS_TRAINING)
create table numbers
(id bigint)
while (#idMax>#idMin)
begin
insert into numbers values(#idMin)
set #idMin=#idMin+1
end
select id from numbers
where id not in (SELECT AG_TRNID FROM PS_TRAINING)

Answer 5

You can create a "virtual" table that contains all numbers from 1 to 999999 then remove those that are present in your table:
select level
from dual
connect by level <= 999999
minus
select ag_trnid
from ps_training;
The above will output all gaps, not just the first one.
The connect by level <= 999999 is an undocumented trick to generate all numbers from 1 to 999999.

Answer 6

Here's one way of finding the start and end numbers of the gaps:
WITH sample_data AS (SELECT 1 ID FROM dual UNION ALL
SELECT 2 ID FROM dual UNION ALL
SELECT 4 ID FROM dual UNION ALL
SELECT 5 ID FROM dual UNION ALL
SELECT 8 ID FROM dual UNION ALL
SELECT 10 ID FROM dual UNION ALL
SELECT 20 ID FROM dual UNION ALL
SELECT 22 ID FROM dual UNION ALL
SELECT 23 ID FROM dual UNION ALL
SELECT 27 ID FROM dual UNION ALL
SELECT 28 ID FROM dual)
-- end of mimicking data in a table called "sample_data"
-- see SQL below:
SELECT prev_id + 1 first_number_in_gap,
ID - 1 last_number_in_gap
FROM (SELECT ID,
LAG(ID, 1, ID - 1) OVER (ORDER BY ID) prev_id
FROM sample_data)
WHERE ID - prev_id > 1;
FIRST_NUMBER_IN_GAP LAST_NUMBER_IN_GAP
------------------- ------------------
3 3
6 7
9 9
11 19
21 21
24 26

Answer 7

In the end I created a table with all possible values and select the minimum from it that doesn't exist in the base table. It works well. I had hoped for a sql statement to avoid creating yet another object but couldn't.
Yes this is Oracle, I neglected to mention.
Thank you all for the suggestions and assistance, it is much appreciated!