I have a set of data like this:
MinNo: 2500
MaxNo: 2700
IncrementStep: 10
Between the minimum number and the maximum number a list of all possible numbers with the given step are to be listed as shown below:
2500
2510
2520
2530
2540
2550
2560
2570
2580
2590
2600
2610
2620
2630
2640
2650
2660
2670
2680
2690
2700
I'm aware that this can be achieved using a while loop. Kindly let me know if this can be done using a select query using Common Table Expressions (if needed). Thanks in advance.
You can use a numbers table (or master..spt_values).
declare #MinNo int
declare #MaxNo int
declare #IncrementStep int
set #MinNo = 2500
set #MaxNo = 2700
set #IncrementStep = 10
select #MinNo + Number * #IncrementStep
from master..spt_values
where type = 'P' and
number between 0 and (#MaxNo - #MinNo) / #IncrementStep
Or a recursive CTE
;with C as
(
select #MinNo as Num
union all
select Num + #IncrementStep
from C
where Num < #MaxNo
)
select Num
from C
See this useful function
CREATE FUNCTION [dbo].[Sequence](#min INT, #max INT, #step INT)
RETURNS #ret TABLE (id INT PRIMARY KEY)
AS
BEGIN
WITH numbers(id) as
(
SELECT #min id
UNION ALL
SELECT id+#step
FROM numbers
WHERE id < #max
)
INSERT #ret
SELECT id FROM Numbers
OPTION(MAXRECURSION 0)
RETURN
END
#Mikael Eriksson already mentioned a numbers table / tally table (search for it online, there are LOTS of possible uses, many DBAs always want a tally table to be present in any system they manage)
I just wanted to share one non-recursive CTE-based "tally table" solution that I saw online the other day, that I think it amazingly elegant for its huge range (4 thousand million logical rows) and general applicability without any database dependencies:
WITH
E00(N) AS (SELECT 1 UNION ALL SELECT 1),
E02(N) AS (SELECT 1 FROM E00 a, E00 b),
E04(N) AS (SELECT 1 FROM E02 a, E02 b),
E08(N) AS (SELECT 1 FROM E04 a, E04 b),
E16(N) AS (SELECT 1 FROM E08 a, E08 b),
E32(N) AS (SELECT 1 FROM E16 a, E16 b),
cteTally(N) AS (SELECT ROW_NUMBER() OVER (ORDER BY N) FROM E32)
SELECT *
FROM cteTally
WHERE N >= 2500
AND N <= 2700
AND N % 10 = 0
I found it here, but I don't know whether that's the original source of this CTE.
The nice thing about it is that you don't need to worry about min, max, or step size, and yet it performs very well in most (one-off) situations. That said, it should NOT be used in any frequently-called business process; Any physical indexed numbers table will always perform better!
EDIT: I just searched a little more for the source of this method (I had missed the stackoverflow link in the article I referenced), and apparently it's originally attributed to Itzik Ben-Gan, from the bottom of page 255 in a book titled "Inside Microsoft SQL Server 2005 - T-SQL Querying" (says Jeff Moden, who I implicitly trust).
Related
There have been quite a number of question regarding ranges in sql however i cant find anything that resembles my use case (most refer to a Tally or numbers table or procedures, something im trying to avoid seeing the elegance in the link below which uses neither)
Im trying to generate numbers between 1 and x, x coming for another table whilst excluding certain numbers from yet another table. (although there is a link between the 2, see the join)
Based on the answer for generating a range of numbers located at https://stackoverflow.com/a/64151448/1161646, im trying to do the following (approximation of the problem not the actual query):
select number
from excluding_numbers as exclusions join ranges range on range.id = exclusions.ranges_id,
(Select 0 + ROW_NUMBER() over (order by (Select null)) as number
from string_split(replicate(' ', range.limit - 1), ' ')) as numbers
where exclusions.number != number
and range.id = 1
The problem is the range.limit in the inner query 'numbers'
Example data:
ranges
id
limit
1
120000
2
10
3
10000000000
excluding_numbers
id
number
ranges_id
1
1
2
2
2
2
3
50000
3
This is saying from the range of 1 to 10 (ranges id 2) exclude number 1 and 2, from ranges 1 to 10000000000 (ranges id 3) only exclude number 50000.
Assuming I understand correctly, and that your numbers are 1+, you could use a Tally table to do this. An inline method would be one method:
WITH N AS(
SELECT N
FROM (VALUES(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL))N(N)),
Tally AS(
SELECT TOP (SELECT x FROM dbo.YourTable) --Table with the number of rows you want. I assume this has one row
ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS I
FROM N N1, N N2, N N3, N N4, N N5, N N6, N N7) --Up to 10,000,000 rows
SELECT T.I
FROM Tally T
WHERE NOT EXISTS (SELECT 1
FROM dbo.ExcludeTable ET--Table with numbers to exclude
WHERE ET.Number = T.I);
You could, however, create an inline table function for your tally, and then reference that instead:
CREATE FUNCTION [dbo].[Tally] (#LastNumber bigint, #Zero bit)
RETURNS table
AS RETURN
WITH N AS(
SELECT N
FROM (VALUES(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL))N(N)),
Tally AS(
SELECT 0 AS I
WHERE #Zero = 0
AND #LastNumber IS NOT NULL --Return 0 rows if NULL
UNION ALL
SELECT TOP (ISNULL(#LastNumber,0)) --Return 0 rows if NULL
ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS I
FROM N N1, N N2, N N3, N N4, N N5, N N6, N N7) --Up to 10,000,000 rows
SELECT I
FROM Tally T;
GO
SELECT T.I
FROM dbo.YourTable YT --Table with the number of rows you want. I assume this has one row
CROSS APPLY dbo.Tally(YT.x,1) T
WHERE NOT EXISTS (SELECT 1
FROM dbo.ExcludeTable ET--Table with numbers to exclude
WHERE ET.Number = T.I);
I need to create a range number from 1 to n.
For example, the parameter is #StartingValue
#StartingValue int = 96
Then the result should be:
Number
-------------
96
95
94
93
92
ff.
1
Does anyone have an idea how to do this?
Thank you.
Use a Tally Table to generate the numbers:
DECLARE #N INT = 96
;WITH E1(N) AS( -- 10 ^ 1 = 10 rows
SELECT 1 FROM(VALUES (1),(1),(1),(1),(1),(1),(1),(1),(1),(1))t(N)
),
E2(N) AS(SELECT 1 FROM E1 a CROSS JOIN E1 b), -- 10 ^ 2 = 100 rows
E4(N) AS(SELECT 1 FROM E2 a CROSS JOIN E2 b), -- 10 ^ 4 = 10,000 rows
E8(N) AS(SELECT 1 FROM E4 a CROSS JOIN E4 b), -- 10 ^ 8 = 10,000,000 rows
CteTally(N) AS(
SELECT TOP(#N) ROW_NUMBER() OVER(ORDER BY(SELECT NULL))
FROM E8
)
SELECT * FROM CteTally ORDER BY N DESC
Explanation taken from Jeff's article (linked above):
The CTE called E1 (as in 10E1 for scientific notation) is nothing more
than ten SELECT 1's returned as a single result set.
E2 does a CROSS JOIN of E1 with itself. That returns a single result
set of 10*10 or up to 100 rows. I say "up to" because if the TOP
function is 100 or less, the CTE's are "smart" enough to know that it
doesn't actually need to go any further and E4 and E8 won't even come
into play. If the TOP has a value of less than 100, not all 100 rows
that E2 is capable of making will be made. It'll always make just
enough according to the TOP function.
You can follow from there. E4 is a CROSS JOIN of E2 and will make up
to 100*100 or 10,000 rows and E8 is a CROSS JOIN of E4 which will make
more rows than most people will ever need. If you do need more, then
just add an E16 as a CROSS JOIN of E8 and change the final FROM clause
to FROM E16.
What's really amazing about this bad-boy is that is produces ZERO
READS. Absolutely none, nada, nil.
One simple method is a numbers table. For a reasonable number (up to the low thousands), you can use spt_values:
with numbers as (
select top 96 row_number() over (order by (select null)) as n
from t
)
. . .
Another method is a recursive CTE:
with numbers as (
select 96 as n
union all
select n - 1
from numbers
where num > 1
)
For larger values, you'll need to use the MAXRECURSION option.
And another way.
SELECT N.number FROM
master..spt_values N
WHERE
N.type = 'P' AND
N.number BETWEEN 1 AND 96
ORDER BY N.number DESC
More details on spt_values What is the purpose of system table master..spt_values and what are the meanings of its values?
Sequance of no's can be generated by following ways:
1. Using row_number by querying a large table and get the sequance.
2. Using system tables as you can see other people comments.
3. Using recursive CTE.
declare #maxValue int = 96
; WITH rangetest AS
(
SELECT MinValue = #maxValue
UNION ALL
SELECT MinValue = MinValue - 1
FROM rangetest
WHERE MinValue > 1
)
SELECT *
from rangetest
OPTION (MAXRECURSION 0)
Assume we have the following table:
id name member
1 jacky a;b;c
2 jason e
3 kate i;j;k
4 alex null
Now I want to use the sql or t-sql to return the following table:
1 jacky a
1 jacky b
1 jacky c
2 jason e
3 kate i
......
How to do that?
I'm using the MSSQL, MYSQL and Oracle database.
This is the shortest and readable string-to-rows splitter one could devise, and could be faster too.
Use case of choosing pure CTE instead of function, e.g. when you're not allowed to create a function on database :-)
Creating rows generator via function(which could be implemented by using loop or via CTE too) shall still need to use lateral joins(DB2 and Sybase have this functionality, using LATERAL keyword; In SQL Server, this is similar to CROSS APPLY and OUTER APPLY) to ultimately join the splitted rows generated by a function to the main table.
Pure CTE approach could be faster than function approach. The speed metrics lies in profiling though, just check the execution plan of this compared to other solutions if this is indeed faster:
with Pieces(theId, pn, start, stop) AS
(
SELECT id, 1, 1, charindex(';', member)
from tbl
UNION ALL
SELECT id, pn + 1, stop + 1, charindex(';', member, stop + 1)
from tbl
join pieces on pieces.theId = tbl.id
WHERE stop > 0
)
select
t.id, t.name,
word =
substring(t.member, p.start,
case WHEN stop > 0 THEN p.stop - p.start
ELSE 512
END)
from tbl t
join pieces p on p.theId = t.id
order by t.id, p.pn
Output:
ID NAME WORD
1 jacky a
1 jacky b
1 jacky c
2 jason e
3 kate i
3 kate j
3 kate k
4 alex (null)
Base logic sourced here: T-SQL: Opposite to string concatenation - how to split string into multiple records
Live test: http://www.sqlfiddle.com/#!3/2355d/1
Well... let me first introduce you to Adam Machanic who taught me about a Numbers table. He's also written a very fast split function using this Numbers table.
http://dataeducation.com/counting-occurrences-of-a-substring-within-a-string/
After you implement a Split function that returns a table, you can then join against it and get the results you want.
IF OBJECT_ID('dbo.Users') IS NOT NULL
DROP TABLE dbo.Users;
CREATE TABLE dbo.Users
(
id INT IDENTITY NOT NULL PRIMARY KEY,
name VARCHAR(50) NOT NULL,
member VARCHAR(1000)
)
GO
INSERT INTO dbo.Users(name, member) VALUES
('jacky', 'a;b;c'),
('jason', 'e'),
('kate', 'i;j;k'),
('alex', NULL);
GO
DECLARE #spliter CHAR(1) = ';';
WITH Base AS
(
SELECT 1 AS n
UNION ALL
SELECT n + 1
FROM Base
WHERE n < CEILING(SQRT(1000)) --generate numbers from 1 to 1000, you may change it to a larger value depending on the member column's length.
)
, Nums AS --Numbers Common Table Expression, if your database version doesn't support it, just create a physical table.
(
SELECT ROW_NUMBER() OVER(ORDER BY (SELECT 0)) AS n
FROM Base AS B1 CROSS JOIN Base AS B2
)
SELECT id,
SUBSTRING(member, n, CHARINDEX(#spliter, member + #spliter, n) - n) AS element
FROM dbo.Users
JOIN Nums
ON n <= DATALENGTH(member) + 1
AND SUBSTRING(#spliter + member, n, 1) = #spliter
ORDER BY id
OPTION (MAXRECURSION 0); --Nums CTE is generated recursively, we don't want to limit recursion count.
I am working on a function that will take a low number and a high number as paramaters and returns a table containing everything between (and including).
I know I could use a cursor and increment a variable adding it to a scope based table every iteration, but I would prefer to avoid a cursor if possible. Does anyone else have a suggestion for a way to do this? (As i'm typing this im thinking possibly a CTE, which I will go investigate).
Yes, you can use a recursive CTE to do this. For example to generate numbers between 10 and 20 inclusive:
WITH f AS
(
SELECT 10 AS x
UNION ALL
SELECT x + 1 FROM f WHERE x < 20
)
SELECT * FROM f
Just create an indexed permanent auxiliary numbers table and be done with it. This will out perform any other method.
See Jeff Moden's answer here for more details and a script to populate such a table. if for some reason that isn't an option this should beat the recursive CTE according to the performance tests in the linked answer.
WITH E00(N) AS (SELECT 1 UNION ALL SELECT 1),
E02(N) AS (SELECT 1 FROM E00 a, E00 b),
E04(N) AS (SELECT 1 FROM E02 a, E02 b),
E08(N) AS (SELECT 1 FROM E04 a, E04 b),
E16(N) AS (SELECT 1 FROM E08 a, E08 b),
E32(N) AS (SELECT 1 FROM E16 a, E16 b),
cteTally(N) AS (SELECT ROW_NUMBER() OVER (ORDER BY N) FROM E32)
SELECT N FROM cteTally
WHERE N BETWEEN 10 AND 20
This could easily be done using code, but I wondered if it could be done at the database level using SQL Server (2008).
I have a table similar to below:
CROP_ID YEAR_ PRODUCTION
1 1 0
1 2 300
1 3 500
2 1 100
2 2 700
I want to be able to run a query to repeat this for n number of years, per crop type e.g.
CROP_ID YEAR_ PRODUCTION
1 1 0
1 2 300
1 3 500
1 4 0
1 5 300
1 6 500
etc.
I'm not sure of the best approach, I presume I would need a SP and pass in a year variable, and use a loop construct? However the exact syntax escapes me. Any help appreciated.
Update
Sorry for not providing all the information in my original post. The table will allow for multiple crop types, and for Produciton values to be updated so Case statements with fixed variables are not really suitable. Apologies for not being clearer.
Update
With the TVF answer I used the following modified SQL to select by CropType for 20 years.
select top 20 b.CROP_ID,
YEAR_ = n.num * (select count() from MyBaseTable where CROP_ID = 3) + b.YEAR,
b.PRODUCTION from MyBaseTable b, dbo.fnMakeNRows(20) n
where CROP_ID = 3
You can do this in standard SQL without creating a stored procedure or using temp tables. The example below will do this for 12 years. You can extend it out to any number of years:
insert into CropYield
(CropID, Year_, Production)
Select 1, a.a + (10 * b.a),
case (a.a + (10 * b.a)) % 3
when 0 then 500
when 1 then 0
when 2 then 300
end
from (Select 0 as a union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) as a
cross join (Select 0 as a union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) as b
where a.a + (10 * b.a) between 1 and 12
You could use a table-valued-function instead of a stored proc, which gives a little more flexibility for what you do with the result (as it can be selected from directly, inserted into another table, joined to other tables, etc).
You could also make this more generic by having a TVF generate N rows (with a number from 0 to N-1 on each row) and then use some simple expressions to generate the columns you need from this. I have found such a TVF to be useful in a variety of situations.
If you need to generate more complicated data than you can with 0..N-1 and simple expressions, then you should create a TVF dedicated to your specific needs.
The following example shows how a generic TVF could be used to generate the data you ask for:
create function fnMakeNRows (#num as integer)
returns #result table (num integer not null) as
begin
if #num is null or #num = 0
begin
return
end
declare #n as integer
set #n = 0
while #n < #num
begin
insert into #result values (#n)
set #n = #n + 1
end
return
end
go
select
CROP_ID = 1,
YEAR_ = num,
PRODUCTION = case num % 3 when 0 then 0 when 1 then 300 else 500 end
from dbo.fnMakeNRows(100000)
You can also use this to duplicate rows in an existing table (which I think is more like what you want). For example, assuming base_table contains the three rows at the beginning of your question, you can turn the 3 rows into 60 rows using the following:
select
b.CROP_ID,
YEAR_ = n.num * (select count(*) from base_table) + b.YEAR_,
b.PRODUCTION
from base_table b, dbo.fnMakeNRows(20) n
This (hopefully) shows the utility of a generic fnMakeNRows function.
A common trick to produce this kind of data without the need of a stored procedure is with the use of a table of constants. Because such a table can be of generic use, it can be created with say all the integers between 1 and 100 or even 1 and 1,000 depending on usage.
For exmaple
CREATE TABLE tblConstNums
( I INT )
INSERT INTO tblConstNums VALUES (1)
INSERT INTO tblConstNums VALUES (2)
INSERT INTO tblConstNums VALUES (3)
INSERT INTO tblConstNums VALUES (4)
INSERT INTO tblConstNums VALUES (5)
-- ...
INSERT INTO tblConstNums VALUES (1000)
The the solution can be written declaratively (without requiring Stored Procedure or more generally procedural statements:
SELECT CROP_ID, YEAR_ * I, PRODUCTION
FROM myCropTable T
JOIN tblConstNums C on 1=1
WHERE I in (1, 2, 3)
order by CROP_ID, YEAR_ * I, PRODUCTION
Note that the table of constants may include several columns for commonly used cases. For example, and even though many of these can be expressed as mathematical expressions of numbers in a basic 0 to n sequence, one can have a column with only even number, another one with odd numbers, another one with multiples of 5 etc. Also if it small enough, no indexes are needed on a table of constants but these may become useful on a bigger on.
use this one:
WITH tn (n) as
(
SELECT 0
UNION ALL
SELECT n+1
FROM tn
WHERE tn.n < 10
)
SELECT DISTINCT t.CROP_ID, t.YEAR_ + (3*tn.n), t.PRODUCTION
FROM table t, tn
/*
WHERE tn.n < 10 ==> you will get 1 -> (10*3) + 3 = 33
*/