Related
so I have a statement I believe should work... However it feels pretty suboptimal and I can't for the life of me figure out how to optimise it.
I have the following tables:
Transactions
[Id] is PRIMARY KEY IDENTITY
[Hash] has a UNIQUE constraint
[BlockNumber] has an Index
Transfers
[Id] is PRIMARY KEY IDENTITY
[TransactionId] is a Foreign Key referencing [Transactions].[Id]
TokenPrices
[Id] is PRIMARY KEY IDENTITY
TokenPriceAttempts
[Id] is PRIMARY KEY IDENTITY
[TransferId] is a Foreign Key referencing [Transfers].[Id]
What I want to do, is select all the transfers, with a few bits of data from their related transaction (one transaction to many transfers), where I don't currently have a price stored in TokenPrices related to that transfer.
In the first part of the query, I am getting a list of all the transfers, and calculating the DIFF between the nearest found token price. If one isn't found, this is null (what I ultimately want to select). I am allowing 3 hours either side of the transaction timestamp - if nothing is found within that timespan, it will be null.
Secondly, I am selecting from this set, ensuring first that diff is null as this means the price is missing, and finally that token price attempts either doesn't have an entry for attempting to get a price, or if it does than it has fewer than 5 attempts listed and the last attempt was more than a week ago.
The way I have laid this out results in essentially 3 of the same / similar SELECT statements within the WHERE clause, which feels hugely suboptimal...
How could I improve this approach?
WITH [transferDateDiff] AS
(
SELECT
[t1].[Id],
[t1].[TransactionId],
[t1].[From],
[t1].[To],
[t1].[Value],
[t1].[Type],
[t1].[ContractAddress],
[t1].[TokenId],
[t2].[Hash],
[t2].[Timestamp],
ABS(DATEDIFF(SECOND, [tp].[Timestamp], [t2].[Timestamp])) AS diff
FROM
[dbo].[Transfers] AS [t1]
LEFT JOIN
[dbo].[Transactions] AS [t2]
ON [t1].[TransactionId] = [t2].[Id]
LEFT JOIN
(
SELECT
*
FROM
[dbo].[TokenPrices]
)
AS [tp]
ON [tp].[ContractAddress] = [t1].[ContractAddress]
AND [tp].[Timestamp] >= DATEADD(HOUR, - 3, [t2].[Timestamp])
AND [tp].[Timestamp] <= DATEADD(HOUR, 3, [t2].[Timestamp])
WHERE
[t1].[Type] < 2
)
SELECT
[tdd].[Id],
[tdd].[TransactionId],
[tdd].[From],
[tdd].[To],
[tdd].[Value],
[tdd].[Type],
[tdd].[ContractAddress],
[tdd].[TokenId],
[tdd].[Hash],
[tdd].[Timestamp]
FROM
[transferDateDiff] AS tdd
WHERE
[tdd].[diff] IS NULL AND
(
(
SELECT
COUNT(*)
FROM
[dbo].[TokenPriceAttempts] tpa
WHERE
[tpa].[TransferId] = [tdd].[Id]
)
= 0 OR
(
(
SELECT
COUNT(*)
FROM
[dbo].[TokenPriceAttempts] tpa
WHERE
[tpa].[TransferId] = [tdd].[Id]
)
< 5 AND
(
DATEDIFF(DAY,
(
SELECT
MAX([tpa].[Created])
FROM
[dbo].[TokenPriceAttempts] tpa
WHERE
[tpa].[TransferId] = [tdd].[Id]
),
CURRENT_TIMESTAMP
) >= 7
)
)
)
Here is an attempt to help simplify. I stripped out all the [brackets] that really are not required unless you are running into something like a reserved keyword, or columns with spaces in their name (bad to begin with).
Anyhow, your main query had 3 instances of a select per ID. To eliminate that, I did a LEFT JOIN to a subquery that pulls all transfers of type < 2 AND JOINS to the price attempts ONCE. This way, the result will have already pre-aggregated the count(*) and Max(Created) done ONCE for the same basis of transfers in question with your WITH CTE declaration. So you dont have to keep running the 3 queries each time, and you dont have to query the entire table of ALL transfers, just those with same underlying type < 2 condition. The result subquery alias "PQ" (preQuery)
This now simplifies the readability of the outer WHERE clause from the redundant counts per Id.
WITH transferDateDiff AS
(
SELECT
t1.Id,
t1.TransactionId,
t1.From,
t1.To,
t1.Value,
t1.Type,
t1.ContractAddress,
t1.TokenId,
t2.Hash,
t2.Timestamp,
ABS( DATEDIFF( SECOND, tp.Timestamp, t2.Timestamp )) AS diff
FROM
dbo.Transfers t1
LEFT JOIN dbo.Transactions t2
ON t1.TransactionId = t2.Id
LEFT JOIN dbo.TokenPrices tp
ON t1.ContractAddress = tp.ContractAddress
AND tp.Timestamp >= DATEADD(HOUR, - 3, t2.Timestamp)
AND tp.Timestamp <= DATEADD(HOUR, 3, t2.Timestamp)
WHERE
t1.Type < 2
)
SELECT
tdd.Id,
tdd.TransactionId,
tdd.From,
tdd.To,
tdd.Value,
tdd.Type,
tdd.ContractAddress,
tdd.TokenId,
tdd.Hash,
tdd.Timestamp
FROM
transferDateDiff tdd
LEFT JOIN
( SELECT
t1.Id,
COUNT(*) Attempts,
MAX(tpa.Created) MaxCreated
FROM
dbo.Transfers t1
JOIN dbo.TokenPriceAttempts tpa
on t1.Id = tpa.TransferId
WHERE
t1.Type < 2
GROUP BY
t1.Id ) PQ
on tdd.Id = PQ.Id
WHERE
tdd.diff IS NULL
AND ( PQ.Attempts IS NULL
OR PQ.Attempts = 0
OR ( PQ.Attempts < 5
AND DATEDIFF(DAY, PQ.MaxCreated, CURRENT_TIMESTAMP ) >= 7
)
)
REVISED to remove the WITH CTE into a single query
SELECT
t1.Id,
t1.TransactionId,
t1.From,
t1.To,
t1.Value,
t1.Type,
t1.ContractAddress,
t1.TokenId,
t2.Hash,
t2.Timestamp
FROM
-- Now, this pre-query is left-joined to token price attempts
-- so ALL Transfers of type < 2 are considered
( SELECT
t1.Id,
coalesce( COUNT(*), 0 ) Attempts,
MAX(tpa.Created) MaxCreated
FROM
dbo.Transfers t1
LEFT JOIN dbo.TokenPriceAttempts tpa
on t1.Id = tpa.TransferId
WHERE
t1.Type < 2
GROUP BY
t1.Id ) PQ
-- Now, we can just directly join to transfers for the rest
JOIN dbo.Transfers t1
on PQ.Id = t1.Id
-- and the rest from the WITH CTE construct
LEFT JOIN dbo.Transactions t2
ON t1.TransactionId = t2.Id
LEFT JOIN dbo.TokenPrices tp
ON t1.ContractAddress = tp.ContractAddress
AND tp.Timestamp >= DATEADD(HOUR, - 3, t2.Timestamp)
AND tp.Timestamp <= DATEADD(HOUR, 3, t2.Timestamp)
WHERE
ABS( DATEDIFF( SECOND, tp.Timestamp, t2.Timestamp )) IS NULL
AND ( PQ.Attempts = 0
OR ( PQ.Attempts < 5
AND DATEDIFF(DAY, PQ.MaxCreated, CURRENT_TIMESTAMP ) >= 7 )
)
I don't understand why you are doing this:
LEFT JOIN
(
SELECT
*
FROM
[dbo].[TokenPrices]
)
AS [tp]
ON [tp].[ContractAddress] = [t1].[ContractAddress]
AND [tp].[Timestamp] >= DATEADD(HOUR, - 3, [t2].[Timestamp])
AND [tp].[Timestamp] <= DATEADD(HOUR, 3, [t2].[Timestamp])
Isn't this a
LEFT JOIN [dbo].[TokenPrices] as TP ...
This:
SELECT
COUNT(*)
FROM
[dbo].[TokenPriceAttempts] tpa
WHERE
[tpa].[TransferId] = [tdd].[Id]
could be another CTE instead of being a sub...
In fact any of your sub queries could be CTE's, that's part of a CTE is making things easier to read.
,TPA
AS
(
SELECT COUNT(*)
FROM [dbo].[TokenPriceAttempts] tpa
WHERE [tpa].[TransferId] = [tdd].[Id]
)
So I might be stuck at something very trivial but can't figure out how to make it work. I create a 2 blocks of code that work in SQL but I have some problems with the date variable declaration in Oracle SQL.
I had write access to the SQL database when I create these codes so I did a 'Insert Into' to create temp tables. I don't have write access anymore. So I am using CTEs for it.
The original code looks like this:
DECLARE #Startdate Datetime = '2021-Jun-01 00:00:00.000'
DECLARE #Enddate Datetime = '2021-Jun-30 00:00:00.000'
Insert into Temp1
select ...
from ...
WHILE Startdate <= Enddate
BEGIN
Insert into Temp2
select ...
from (Temp 1)
left join
select ...
set #startdate=dateadd(d,1,#startdate)
end;
With my new code, I have made the following adjustmnets:
VARIABLE Startdate Datetime = '2021-Jun-01 00:00:00.000'
VARIABLE Enddate Datetime = '2021-Jun-30 00:00:00.000'
EXEC :Startdate := '2021-Jun-30 00:00:00.000'
EXEC :Enddate := '2021-Jun-30 00:00:00.000'
WITH Temp1 as (
select ...
from ...),
/* Unsure about using WHILE with with 2 CTEs so removing them for now but will need to be added*/
WITH Temp2 as
select ...
from (Temp 1)
left join
select ...
set startdate = :startdate + 1
end)
select * from Temp2;
The 2 blocks of code work perfectly individually. I think my concern lies with one or all of the following:
Variable Declaration - I read a couple of stackoverflow posts and it seems like there is binding variable and substitution variable. Is there a different way to declare variables?
The WHILE Loop specially between 2 CTEs. Can we do a while loop as a CTE? (similar to this) create while loop with cte
How the date is incremented. Is this the proper way to increment dates in Oracle PL/SQL?
Any guidance would be helpful.
Also adding 2 blocks of codes for reference:
Details of Tables:
Transactions - Contains Transaction information. Execution Date is a timestamp of the transaction execution
Account - Contains Account Information with a unique Account_Key for every account
Code_Rel - Maps the transaction code to a transaction type
Group Rel - Maps the transaction type to a transaction group
/***Block 1 of Code***/
insert into Temp1
select
a.ACCOUNT_KEY
,a.SPG_CD
,t.EXECUTION_DATE
from Schema_Name.TRANSACTIONS t
inner join Schema_Name.ACCOUNT a on a.en_sk=t.ac_sk
inner join Schema_Name.Code_Rel tr on t.t_cd_s = tr.t_cd_s
inner join ( select * from Schema_Name.Group_Rel
where gtrt_cd in ('Type1','Type2')) tt on tr.trt_cd = tt.trt_cd
where t.EXECUTION_DATE >= #startdate and t.EXECUTION_DATE<=#EndDt
and tt.gtrt_cd in ('Type1','Type2')
group by a.ACCOUNT_KEY ,a.SPG_CD, t.EXECUTION_DATE;
/***WHILE LOOP***/
while #startdate <= #EndDt
BEGIN
/***INSERT AND BLOCK 2 OF CODE***/
insert into Temp2
select table1.account_key, table1.SPG_CD, #startdate, coalesce(table2.sum_tr1,0),coalesce(table3.sum_tr2,0),
case when coalesce(table3.sum_tr2,0)>0 THEN coalesce(table2.sum_tr1,0)/coalesce(table3.sum_tr2,0) ELSE 0 END,
case when coalesce(table3.sum_tr2,0)>0 THEN
CASE WHEN coalesce(table2.sum_tr1,0)/coalesce(table3.sum_tr2,0)>=0.9 and coalesce(table2.sum_tr1,0)/coalesce(table3.sum_tr2,0)<=1.10 and coalesce(table2.sum_tr1,0)>=1000 THEN 'Yes' else 'No' END
ELSE 'No' END
FROM ( SELECT * FROM Temp1 WHERE execution_date=#startdate) TABLE1 LEFT JOIN
(
select a.account_key,a.SPG_CD, SUM(t.AC_Amt) as sum_tr1
from Schema_Name.TRANSACTIONS t
inner join Schema_Name.ACCOUNT a on a.en_sk=t.ac_sk
inner join Schema_Name.Code_Rel tr on t.t_cd_s = tr.t_cd_s
inner join ( select * from Schema_Name.Group_Rel
where gtrt_cd in ('Type1')) tt on tr.trt_cd = tt.trt_cd
where t.EXECUTION_DATE <= #startdate
and t.EXECUTION_DATE >=dateadd(day,-6,#startdate)
and tt.gtrt_cd in ('Type1')
group by a.account_key, a.SPG_CD
) table2 ON table1.account_key=table2.account_key
LEFT JOIN
(
select a.account_key,a.SPG_CD, SUM(t.AC_Amt) as sum_tr2
from Schema_Name.TRANSACTIONS t
inner join Schema_Name.ACCOUNT a on a.en_sk=t.ac_sk
inner join Schema_Name.Code_Rel tr on t.t_cd_s = tr.t_cd_s
inner join ( select * from Schema_Name.Group_Rel
where gtrt_cd in ('Type2')) tt on tr.trt_cd = tt.trt_cd
where t.EXECUTION_DATE <= #startdate
and t.EXECUTION_DATE >=dateadd(day,-6,#startdate)
and tt.gtrt_cd in ('Type2')
group by a.account_key, a.SPG_CD ) table3 on table1.account_key=table3.account_key
where coalesce(table2.sum_tr1,0)>=1000
set #startdate=dateadd(d,1,#startdate)
end;
You do not need to use PL/SQL or a WHILE loop or to declare variables and can probably do it all in a single SQL query using subquery factoring clauses (and recursion) to generate a calendar of incrementing dates. Something like this made-up example:
INSERT INTO temp2 (col1, col2, col3)
WITH time_bounds(start_date, end_date) AS (
-- You can declare the bounds in the query.
SELECT DATE '2021-06-01',
DATE '2021-06-30'
FROM DUAL
),
calendar (dt, end_date) AS (
-- Recursive query to generate a row for each day.
SELECT start_date, end_date FROM time_bounds
UNION ALL
SELECT dt + INTERVAL '1' DAY, end_date
FROM calendar
WHERE dt + INTERVAL '1' DAY <= end_date
),
temp1 (a, b, c) AS (
-- Made-up query
SELECT a, b, c FROM some_table
),
temp2 (a, d, e) AS (
-- Another made-up query.
SELECT t1.a,
s2.d,
s2.e
FROM temp1 t1
LEFT OUTER JOIN some_other_table s2
ON (t1.b = s2.b)
)
-- Get the values to insert.
SELECT t2.a,
t2.d,
t2.e
FROM temp2 t2
INNER JOIN calendar c
ON (t2.e = c.dt)
WHERE a BETWEEN 3.14159 AND 42;
If you try doing it with multiple inserts in a PL/SQL loop then it will be much slower than a single statement.
Suppose that I have a relation with only 1 column "Value (INT)" and its values are in descending order.
+----------+
+ VALUE +
+----------+
+ 10 +
+ 9 +
+ 8 +
+ 7 +
....
How can list all the combinations which contains two tuples such that the first tuple is greater than the second tuple
Note: It may exist two tuples with same value
The desired outputs should be like: (10,9) (10, 8) (9,8), (9,7) (8,7)
You can do a cross join on the same table.
SELECT t1.VALUE AS VALUE1, t2.VALUE AS VALUE2
FROM thing t1 JOIN thing t2 ON (t1.VALUE != t2.VALUE AND t1.VALUE > t2.VALUE)
I understand that a single tuple may apear only twice on the left side of the resultset, am I right? That's why there is no (10,7)?
Then you need to compute row number.
select t1.value, t2.value
from
(
select t.value, row_number(order by t.value) as rnum
from table t
) t1 inner join
(
select t.value, row_number(order by t.value) as rnum
from table t
) t2 on t1.value > t2.value and t1.rnum < t2.rnum + 2
Performance of this query will be pretty bad, but I don't know what database are you using - I've used MS SQL row_number function.
Another idea:
If you are using SQL Server 2012+ and your answer to the question posed at the begining of this post is positive, you can use:
select t.value, lead(t.value,1,0) over(order by t.value desc) as lead1
lead(t.value,2,0) over(order by t.value desc) as lead2
from table t
You may need to handle 0 (defulat value if there is no "lead" tuple). I'm not sure if output in this form is acceptable.
And here you go with cursor solution:
DECLARE #result TABLE
(
value1 int,
value2 int
);
DECLARE
#value int,
#lag1 int,
#lag2 int
DECLARE c CURSOR LOCAL STATIC FORWARD_ONLY READ_ONLY
FOR
SELECT value
FROM table
ORDER BY value desc
OPEN c;
FETCH NEXT FROM c INTO #lag2;
FETCH NEXT FROM c INTO #lag1;
FETCH NEXT FROM c INTO #value;
WHILE ##FETCH_STATUS = 0
BEGIN
INSERT #result(value1, value2) SELECT #lag2, #lag1
INSERT #result(value1, value2) SELECT #lag2, #value
SET #lag2 = #lag1
SET #lag1 = #value
FETCH NEXT FROM c INTO #value
END
CLOSE c;
Again, I used MS SQL syntax. If you write how you want duplicates handled, I can update the solution.
I'm trying to create a query that only returns rows with objects that have three or more orders in a week's timeframe and are only orders submitted after 9/1/13.
SELECT OrderID, DateSubmitted, ObjectID = i.ObjectID
FROM dbo.Object i
JOIN dbo.Order j
ON i.ObjectID = j.ObjectID
WHERE DateSubmitted >= '9/1/2013'
I just can't figure out how to narrow the results to those objects with three or more orders in a week. I've tried numerous GROUP BY and HAVING clauses with no luck. Any help would be greatly appreciated.
Try:
SELECT ObjectID
FROM dbo.Object i
JOIN dbo.Order j ON J.ObjectID = i.ObjectID
WHERE DateSubmitted >= '9/1/2013'
GROUP BY ObjectID
HAVING COUNT(1) >=3
not sure but i need more info on the tables
best guess is in sql
SELECT count(OrderID), i.ObjectID
FROM dbo.Object i
JOIN dbo.Order j
ON i.ObjectID = j.ObjectID
group by i.ObjectID
having DateSubmitted >= '9/1/2013' and count(OrderID)>2
based on your last comments the query you looking for is very simple. use the DatePart function and find out WEEK of that object's order-date. check the below query. also unless you data base is configured already by default Sunday(int 7) is the First Day set in the SQL server. so you for this query purpose you need to set Monday(int 1) as the first day of Week.
you can check the current setting by executing SELECT ##DATEFIRST
if same object has places 3 or more times for multiple weeks below query is returning that object for each of that week. if you needed only Objects then you can skip weekNumber from result set and do Distinct ObjectID.
Do not forget to Reset the DATEFIRST setting to its original Value at the END.
DECLARE #Object TABLE
(
objectID INT
)
DECLARE #Order TABLE
(
orderID INT
,objectID INT
,DateSubmitted DATE
)
INSERT INTO #Object( objectID )
SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
INSERT INTO #Order ( orderID, objectID, DateSubmitted )
SELECT 1,1,'10/2/2013'
UNION ALL SELECT 2,1,'10/3/2013'
UNION ALL SELECT 3,1,'10/5/2013'
UNION ALL SELECT 4,1,'10/09/2013'
UNION ALL SELECT 5,1,'10/10/2013'
UNION ALL SELECT 6,1,'10/13/2013'
UNION ALL SELECT 4,2,'10/15/2013'
UNION ALL SELECT 5,2,'10/16/2013'
UNION ALL SELECT 6,2,'10/21/2013'
UNION ALL SELECT 7,3,'09/02/2013'
UNION ALL SELECT 8,3,'09/03/2013'
UNION ALL SELECT 9,3,'09/04/2013'
DECLARE #CurrentDateFirst INT=##DATEFIRST
SET DATEFIRST 1;
SELECT i.objectID,DATEPART(week,DateSubmitted) AS weekNumber
FROM #Object i
JOIN #Order j
ON i.ObjectID = j.ObjectID
WHERE DateSubmitted >= '9/1/2013'
GROUP BY i.objectID,DATEPART(week,DateSubmitted)
HAVING(COUNT(DISTINCT orderID) >= 3)
ORDER BY i.objectID
SET DATEFIRST #CurrentDateFirst
Convoluted, but I think this will get you what you want...
With WeekSets As
(
Select i.ObjectID,
j.DateSubmitted,
j2.OrderID
From dbo.Object i
JOIN dbo.Order j
ON i.ObjectID = j.ObjectID
JOIN dbo.Order j2
On j.DateSubmitted <= j2.DateSubmitted
And j2.DateSubmitted < DateAdd(Day,7,j.DateSubmitted)
And j.ObjectID = j2.ObjectID
WHERE j.DateSubmitted >= '9/1/2013'
), GroupsOfThree As
(
Select ObjectID,
DateSubmitted
From WeekSets
Group By ObjectID,
DateSubmitted
Having Count(OrderID) >= 3
)
Select Distinct j.OrderID,
j.DateSubmitted,
w.ObjectID
From GroupsOfThree g
Join WeekSets w
On g.ObjectID = w.ObjectID
And g.DateSubmitted = w.DateSubmitted
Join dbo.Order j
On w.ObjectID = j.ObjectID
And w.OrderID = j.OrderID
I'd like to find the first "gap" in a counter column in an SQL table. For example, if there are values 1,2,4 and 5 I'd like to find out 3.
I can of course get the values in order and go through it manually, but I'd like to know if there would be a way to do it in SQL.
In addition, it should be quite standard SQL, working with different DBMSes.
In MySQL and PostgreSQL:
SELECT id + 1
FROM mytable mo
WHERE NOT EXISTS
(
SELECT NULL
FROM mytable mi
WHERE mi.id = mo.id + 1
)
ORDER BY
id
LIMIT 1
In SQL Server:
SELECT TOP 1
id + 1
FROM mytable mo
WHERE NOT EXISTS
(
SELECT NULL
FROM mytable mi
WHERE mi.id = mo.id + 1
)
ORDER BY
id
In Oracle:
SELECT *
FROM (
SELECT id + 1 AS gap
FROM mytable mo
WHERE NOT EXISTS
(
SELECT NULL
FROM mytable mi
WHERE mi.id = mo.id + 1
)
ORDER BY
id
)
WHERE rownum = 1
ANSI (works everywhere, least efficient):
SELECT MIN(id) + 1
FROM mytable mo
WHERE NOT EXISTS
(
SELECT NULL
FROM mytable mi
WHERE mi.id = mo.id + 1
)
Systems supporting sliding window functions:
SELECT -- TOP 1
-- Uncomment above for SQL Server 2012+
previd
FROM (
SELECT id,
LAG(id) OVER (ORDER BY id) previd
FROM mytable
) q
WHERE previd <> id - 1
ORDER BY
id
-- LIMIT 1
-- Uncomment above for PostgreSQL
Your answers all work fine if you have a first value id = 1, otherwise this gap will not be detected. For instance if your table id values are 3,4,5, your queries will return 6.
I did something like this
SELECT MIN(ID+1) FROM (
SELECT 0 AS ID UNION ALL
SELECT
MIN(ID + 1)
FROM
TableX) AS T1
WHERE
ID+1 NOT IN (SELECT ID FROM TableX)
There isn't really an extremely standard SQL way to do this, but with some form of limiting clause you can do
SELECT `table`.`num` + 1
FROM `table`
LEFT JOIN `table` AS `alt`
ON `alt`.`num` = `table`.`num` + 1
WHERE `alt`.`num` IS NULL
LIMIT 1
(MySQL, PostgreSQL)
or
SELECT TOP 1 `num` + 1
FROM `table`
LEFT JOIN `table` AS `alt`
ON `alt`.`num` = `table`.`num` + 1
WHERE `alt`.`num` IS NULL
(SQL Server)
or
SELECT `num` + 1
FROM `table`
LEFT JOIN `table` AS `alt`
ON `alt`.`num` = `table`.`num` + 1
WHERE `alt`.`num` IS NULL
AND ROWNUM = 1
(Oracle)
The first thing that came into my head. Not sure if it's a good idea to go this way at all, but should work. Suppose the table is t and the column is c:
SELECT
t1.c + 1 AS gap
FROM t as t1
LEFT OUTER JOIN t as t2 ON (t1.c + 1 = t2.c)
WHERE t2.c IS NULL
ORDER BY gap ASC
LIMIT 1
Edit: This one may be a tick faster (and shorter!):
SELECT
min(t1.c) + 1 AS gap
FROM t as t1
LEFT OUTER JOIN t as t2 ON (t1.c + 1 = t2.c)
WHERE t2.c IS NULL
This works in SQL Server - can't test it in other systems but it seems standard...
SELECT MIN(t1.ID)+1 FROM mytable t1 WHERE NOT EXISTS (SELECT ID FROM mytable WHERE ID = (t1.ID + 1))
You could also add a starting point to the where clause...
SELECT MIN(t1.ID)+1 FROM mytable t1 WHERE NOT EXISTS (SELECT ID FROM mytable WHERE ID = (t1.ID + 1)) AND ID > 2000
So if you had 2000, 2001, 2002, and 2005 where 2003 and 2004 didn't exist, it would return 2003.
The following solution:
provides test data;
an inner query that produces other gaps; and
it works in SQL Server 2012.
Numbers the ordered rows sequentially in the "with" clause and then reuses the result twice with an inner join on the row number, but offset by 1 so as to compare the row before with the row after, looking for IDs with a gap greater than 1. More than asked for but more widely applicable.
create table #ID ( id integer );
insert into #ID values (1),(2), (4),(5),(6),(7),(8), (12),(13),(14),(15);
with Source as (
select
row_number()over ( order by A.id ) as seq
,A.id as id
from #ID as A WITH(NOLOCK)
)
Select top 1 gap_start from (
Select
(J.id+1) as gap_start
,(K.id-1) as gap_end
from Source as J
inner join Source as K
on (J.seq+1) = K.seq
where (J.id - (K.id-1)) <> 0
) as G
The inner query produces:
gap_start gap_end
3 3
9 11
The outer query produces:
gap_start
3
Inner join to a view or sequence that has a all possible values.
No table? Make a table. I always keep a dummy table around just for this.
create table artificial_range(
id int not null primary key auto_increment,
name varchar( 20 ) null ) ;
-- or whatever your database requires for an auto increment column
insert into artificial_range( name ) values ( null )
-- create one row.
insert into artificial_range( name ) select name from artificial_range;
-- you now have two rows
insert into artificial_range( name ) select name from artificial_range;
-- you now have four rows
insert into artificial_range( name ) select name from artificial_range;
-- you now have eight rows
--etc.
insert into artificial_range( name ) select name from artificial_range;
-- you now have 1024 rows, with ids 1-1024
Then,
select a.id from artificial_range a
where not exists ( select * from your_table b
where b.counter = a.id) ;
This one accounts for everything mentioned so far. It includes 0 as a starting point, which it will default to if no values exist as well. I also added the appropriate locations for the other parts of a multi-value key. This has only been tested on SQL Server.
select
MIN(ID)
from (
select
0 ID
union all
select
[YourIdColumn]+1
from
[YourTable]
where
--Filter the rest of your key--
) foo
left join
[YourTable]
on [YourIdColumn]=ID
and --Filter the rest of your key--
where
[YourIdColumn] is null
For PostgreSQL
An example that makes use of recursive query.
This might be useful if you want to find a gap in a specific range
(it will work even if the table is empty, whereas the other examples will not)
WITH
RECURSIVE a(id) AS (VALUES (1) UNION ALL SELECT id + 1 FROM a WHERE id < 100), -- range 1..100
b AS (SELECT id FROM my_table) -- your table ID list
SELECT a.id -- find numbers from the range that do not exist in main table
FROM a
LEFT JOIN b ON b.id = a.id
WHERE b.id IS NULL
-- LIMIT 1 -- uncomment if only the first value is needed
My guess:
SELECT MIN(p1.field) + 1 as gap
FROM table1 AS p1
INNER JOIN table1 as p3 ON (p1.field = p3.field + 2)
LEFT OUTER JOIN table1 AS p2 ON (p1.field = p2.field + 1)
WHERE p2.field is null;
I wrote up a quick way of doing it. Not sure this is the most efficient, but gets the job done. Note that it does not tell you the gap, but tells you the id before and after the gap (keep in mind the gap could be multiple values, so for example 1,2,4,7,11 etc)
I'm using sqlite as an example
If this is your table structure
create table sequential(id int not null, name varchar(10) null);
and these are your rows
id|name
1|one
2|two
4|four
5|five
9|nine
The query is
select a.* from sequential a left join sequential b on a.id = b.id + 1 where b.id is null and a.id <> (select min(id) from sequential)
union
select a.* from sequential a left join sequential b on a.id = b.id - 1 where b.id is null and a.id <> (select max(id) from sequential);
https://gist.github.com/wkimeria/7787ffe84d1c54216f1b320996b17b7e
Here is an alternative to show the range of all possible gap values in portable and more compact way :
Assume your table schema looks like this :
> SELECT id FROM your_table;
+-----+
| id |
+-----+
| 90 |
| 103 |
| 104 |
| 118 |
| 119 |
| 120 |
| 121 |
| 161 |
| 162 |
| 163 |
| 185 |
+-----+
To fetch the ranges of all possible gap values, you have the following query :
The subquery lists pairs of ids, each of which has the lowerbound column being smaller than upperbound column, then use GROUP BY and MIN(m2.id) to reduce number of useless records.
The outer query further removes the records where lowerbound is exactly upperbound - 1
My query doesn't (explicitly) output the 2 records (YOUR_MIN_ID_VALUE, 89) and (186, YOUR_MAX_ID_VALUE) at both ends, that implicitly means any number in both of the ranges hasn't been used in your_table so far.
> SELECT m3.lowerbound + 1, m3.upperbound - 1 FROM
(
SELECT m1.id as lowerbound, MIN(m2.id) as upperbound FROM
your_table m1 INNER JOIN your_table
AS m2 ON m1.id < m2.id GROUP BY m1.id
)
m3 WHERE m3.lowerbound < m3.upperbound - 1;
+-------------------+-------------------+
| m3.lowerbound + 1 | m3.upperbound - 1 |
+-------------------+-------------------+
| 91 | 102 |
| 105 | 117 |
| 122 | 160 |
| 164 | 184 |
+-------------------+-------------------+
select min([ColumnName]) from [TableName]
where [ColumnName]-1 not in (select [ColumnName] from [TableName])
and [ColumnName] <> (select min([ColumnName]) from [TableName])
Here is standard a SQL solution that runs on all database servers with no change:
select min(counter + 1) FIRST_GAP
from my_table a
where not exists (select 'x' from my_table b where b.counter = a.counter + 1)
and a.counter <> (select max(c.counter) from my_table c);
See in action for;
PL/SQL via Oracle's livesql,
MySQL via sqlfiddle,
PostgreSQL via sqlfiddle
MS Sql via sqlfiddle
It works for empty tables or with negatives values as well. Just tested in SQL Server 2012
select min(n) from (
select case when lead(i,1,0) over(order by i)>i+1 then i+1 else null end n from MyTable) w
If You use Firebird 3 this is most elegant and simple:
select RowID
from (
select `ID_Column`, Row_Number() over(order by `ID_Column`) as RowID
from `Your_Table`
order by `ID_Column`)
where `ID_Column` <> RowID
rows 1
-- PUT THE TABLE NAME AND COLUMN NAME BELOW
-- IN MY EXAMPLE, THE TABLE NAME IS = SHOW_GAPS AND COLUMN NAME IS = ID
-- PUT THESE TWO VALUES AND EXECUTE THE QUERY
DECLARE #TABLE_NAME VARCHAR(100) = 'SHOW_GAPS'
DECLARE #COLUMN_NAME VARCHAR(100) = 'ID'
DECLARE #SQL VARCHAR(MAX)
SET #SQL =
'SELECT TOP 1
'+#COLUMN_NAME+' + 1
FROM '+#TABLE_NAME+' mo
WHERE NOT EXISTS
(
SELECT NULL
FROM '+#TABLE_NAME+' mi
WHERE mi.'+#COLUMN_NAME+' = mo.'+#COLUMN_NAME+' + 1
)
ORDER BY
'+#COLUMN_NAME
-- SELECT #SQL
DECLARE #MISSING_ID TABLE (ID INT)
INSERT INTO #MISSING_ID
EXEC (#SQL)
--select * from #MISSING_ID
declare #var_for_cursor int
DECLARE #LOW INT
DECLARE #HIGH INT
DECLARE #FINAL_RANGE TABLE (LOWER_MISSING_RANGE INT, HIGHER_MISSING_RANGE INT)
DECLARE IdentityGapCursor CURSOR FOR
select * from #MISSING_ID
ORDER BY 1;
open IdentityGapCursor
fetch next from IdentityGapCursor
into #var_for_cursor
WHILE ##FETCH_STATUS = 0
BEGIN
SET #SQL = '
DECLARE #LOW INT
SELECT #LOW = MAX('+#COLUMN_NAME+') + 1 FROM '+#TABLE_NAME
+' WHERE '+#COLUMN_NAME+' < ' + cast( #var_for_cursor as VARCHAR(MAX))
SET #SQL = #sql + '
DECLARE #HIGH INT
SELECT #HIGH = MIN('+#COLUMN_NAME+') - 1 FROM '+#TABLE_NAME
+' WHERE '+#COLUMN_NAME+' > ' + cast( #var_for_cursor as VARCHAR(MAX))
SET #SQL = #sql + 'SELECT #LOW,#HIGH'
INSERT INTO #FINAL_RANGE
EXEC( #SQL)
fetch next from IdentityGapCursor
into #var_for_cursor
END
CLOSE IdentityGapCursor;
DEALLOCATE IdentityGapCursor;
SELECT ROW_NUMBER() OVER(ORDER BY LOWER_MISSING_RANGE) AS 'Gap Number',* FROM #FINAL_RANGE
Found most of approaches run very, very slow in mysql. Here is my solution for mysql < 8.0. Tested on 1M records with a gap near the end ~ 1sec to finish. Not sure if it fits other SQL flavours.
SELECT cardNumber - 1
FROM
(SELECT #row_number := 0) as t,
(
SELECT (#row_number:=#row_number+1), cardNumber, cardNumber-#row_number AS diff
FROM cards
ORDER BY cardNumber
) as x
WHERE diff >= 1
LIMIT 0,1
I assume that sequence starts from `1`.
If your counter is starting from 1 and you want to generate first number of sequence (1) when empty, here is the corrected piece of code from first answer valid for Oracle:
SELECT
NVL(MIN(id + 1),1) AS gap
FROM
mytable mo
WHERE 1=1
AND NOT EXISTS
(
SELECT NULL
FROM mytable mi
WHERE mi.id = mo.id + 1
)
AND EXISTS
(
SELECT NULL
FROM mytable mi
WHERE mi.id = 1
)
DECLARE #Table AS TABLE(
[Value] int
)
INSERT INTO #Table ([Value])
VALUES
(1),(2),(4),(5),(6),(10),(20),(21),(22),(50),(51),(52),(53),(54),(55)
--Gaps
--Start End Size
--3 3 1
--7 9 3
--11 19 9
--23 49 27
SELECT [startTable].[Value]+1 [Start]
,[EndTable].[Value]-1 [End]
,([EndTable].[Value]-1) - ([startTable].[Value]) Size
FROM
(
SELECT [Value]
,ROW_NUMBER() OVER(PARTITION BY 1 ORDER BY [Value]) Record
FROM #Table
)AS startTable
JOIN
(
SELECT [Value]
,ROW_NUMBER() OVER(PARTITION BY 1 ORDER BY [Value]) Record
FROM #Table
)AS EndTable
ON [EndTable].Record = [startTable].Record+1
WHERE [startTable].[Value]+1 <>[EndTable].[Value]
If the numbers in the column are positive integers (starting from 1) then here is how to solve it easily. (assuming ID is your column name)
SELECT TEMP.ID
FROM (SELECT ROW_NUMBER() OVER () AS NUM FROM 'TABLE-NAME') AS TEMP
WHERE ID NOT IN (SELECT ID FROM 'TABLE-NAME')
ORDER BY 1 ASC LIMIT 1
SELECT ID+1 FROM table WHERE ID+1 NOT IN (SELECT ID FROM table) ORDER BY 1;