I want to expand a table using cross apply with a list of number as input.
For example, the original table would look like:
ID
A
B
c
Then, I have a list of random number of unknown length to be passed as a variable, like 1, 2, 3. After this, the table would look like this:
ID
N
A
1
A
2
A
3
B
1
B
2
B
3
C
1
C
2
C
3
I tried to do this in Python and then upload the dataframe to database, but the data inserting part took extremely long as the table have millions of rows.
I was wondering if there is an optimal way to do this purely in SSMS and use SQL only?
Rather than a CROSS APPLY, perhaps a CROSS JOIN will do.
Example
Declare #YourTable table (ID varchar(25))
Insert Into #YourTable values
('A'),('B'),('C')
Declare #String varchar(max)='1,2,3'
Select *
From #YourTable
Cross Join string_split(#String,',')
Results
ID value
A 1
A 2
A 3
B 1
B 2
B 3
C 1
C 2
C 3
You can create a Stored procedure (SP) and call the procedure in Python code. Then you need to provide a list of integers to the SQL stored procedure as an input. String_split function can be used if you are using SQL server 2016 or above. Else you have to write a function to split the string.
You can call this in python function. Get a input values to a list and then assign the list to a string variable and pass the variable to SP as a parameter.
--EXEC Insert_data '1,2,3,4'
CREATE PROCEDURE Insert_data(
-- Add the parameters for the stored procedure here
#Input_list AS VARCHAR(MAX)
)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
SELECT
*
INTO #Temp_integers
FROM
string_split(#Input_list,',')
DROP TABLE IF EXISTS Input_Values
SELECT A.ID
, CAST(B.[value] AS INT) AS [Value]
INTO Input_Values
FROM Temp_values A
CROSS JOIN #Temp_integers B
SELECT *
FROM Input_Values
ORDER BY ID,[Value]
END
GO
Related
Wasn't sure how to title the question but hopefully this makes sense :)
I have a table (OldTable) with an index and a column of comma separated lists. I'm trying to split the strings in the list column and create a new table with the indexes coupled with each of the sub strings of the string it was connected to in the old table.
Example:
OldTable
index | list
1 | 'a,b,c'
2 | 'd,e,f'
NewTable
index | letter
1 | 'a'
1 | 'b'
1 | 'c'
2 | 'd'
2 | 'e'
2 | 'f'
I have created a function that will split the string and return each sub string as a record in a 1 column table as so:
SELECT * FROM Split('a,b,c', ',', 1)
Which will result in:
Result
index | string
1 | 'a'
1 | 'b'
1 | 'c'
I was hoping that I could use this function as so:
SELECT * FROM Split((SELECT * FROM OldTable), ',')
And then use the id and string columns from OldTable in my function (by re-writing it slightly) to create NewTable. But I as far as I understand sending tables into the function doesn't work as I get: "Subquery returned more than 1 value. ... not premitted ... when the subquery is used as an expression."
One solution I was thinking of would be to run the function, as is, on all the rows of OldTable and insert the result of each call into NewTable. But I'm not sure how to iterate each row without a function. And I can't send tables into the a function to iterate so I'm back at square one.
I could do it manually but OldTable contains a few records (1000 or so) so it seems like automation would be preferable.
Is there a way to either:
Iterate over OldTable row by row, run the row through Split(), add the result to NewTable for all rows in OldTable. Either by a function or through regular sql-transactions
Re-write Split() to take a table variable after all
Get rid of the function altogether and just do it in sql transactions?
I'd prefer to not use procedures (don't know if there is a solutions with them either) mostly because I don't want the functionality inside of the DB to be exposed to the outside. If, however that is the "best"/only way to go I'll have to consider it. I'm quite (read very) new to SQL so it might be a needless worry.
Here is my Split() function if it is needed:
CREATE FUNCTION Split (
#string nvarchar(4000),
#delimitor nvarchar(10),
#indexint = 0
)
RETURNS #splitTable TABLE (id int, string nvarchar(4000) NOT NULL) AS
BEGIN
DECLARE #startOfSubString smallint;
DECLARE #endOfSubString smallint;
SET #startOfSubString = 1;
SET #endOfSubString = CHARINDEX(#delimitor, #string, #startOfSubString);
IF (#endOfSubString <> 0)
WHILE #endOfSubString > 0
BEGIN
INSERT INTO #splitTable
SELECT #index, SUBSTRING(#string, #startOfSubString, #endOfSubString - #startOfSubString);
SET #startOfSubString = #endOfSubString+1;
SET #endOfSubString = CHARINDEX(#delimitor, #string, #startOfSubString);
END;
INSERT INTO #splitTable
SELECT #index, SUBSTRING(#string, #startOfSubString, LEN(#string)-#startOfSubString+1);
RETURN;
END
Hope my problem and attempt was explained and possible to understand.
You are looking for cross apply:
SELECT t.index, s.item
FROM OldTable t CROSS APPLY
(dbo.split(t.list, ',')) s(item);
Inserting in the new table just requires an insert or select into clause.
What would be the best approach in SQL Server 2008 to select something that can contain 10 list of data, then compare that data with a specific value in one of it's columns
So something like this below
SELECT bType FROM WORK_STATION WHERE nFileId = 123456789
Which could return either 1 - 10 values MAX (will return at least one value). Then to compare the data from that SQL statement above that we just selected to a specific value to something like
if bType = 1
--DO something
What is the best approach of doing something like this?
declare #table as table(btype int)
declare #btype int
insert into #table
SELECT bType FROM WORK_STATION WHERE nFileId = 123456789
while(exists(select top 1 'x' from #table)) --as long as #table contains records continue
begin
select top 1 #btype = btype from #table
if(#btype = 10)
print 'something'
delete top (1) from #table --remove the previously processed row. also ensures no infinite loop
end
I think you can use SP to declare variables and then compare it with the resultset, if you know that you have only 10 values you can use temp table and insert 10 values.
I hope this is helpful.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Parameterizing an SQL IN clause?
SQL:Casting a String to IDS with IN clause
I want to use a declare local variable in a WHERE IN clause
Something like this:
TABLE XYZ
COL1 COL2
1 A
2 B
3 C
4 D
5 E
RESULT
1 A
2 B
5 E
QUERY
DECLARE #VAR VARCHAR(MAX)
SET #VAR = '1,2,5'
SELECT * FROM XYZ WHERE COL1 IN #VAR
How do I do this?
Note :
I cant have a server function
I can only create primitive value (by code) and use it with my query.
I search a way that I will only change my var and not the query itself.
my code look like:
list (dbparameter)
mylist.add ('#var','1,2,5')
commandsql.returnresult(myQueryInString,mylist)
I want to
DECLARE #var TABLE
(
value INT
)
INSERT
INTO #var
VALUES
(1), (3), (5)
/* You would need three separate inserts in 2005 */
SELECT *
FROM xyz
WHERE col1 IN
(
SELECT value
FROM #var
)
You can also write a table-valued function which splits a CSV, but if your client library supports passing table variables, this is a preferred option.
You can find the function definition in the Erlang Sommarskog's article (search for simple_intlist_to_tbl). Declare it and call like this:
DECLARE #var VARCHAR(100) = '1,3,5'
SELECT *
FROM xyz
WHERE col1 IN
(
SELECT number
FROM simple_intlist_to_tbl(#var)
)
If your query is more complex than that, you would want to materialize this list first:
DECLARE #var VARCHAR(100) = '1,3,5'
DECLARE #tvar TABLE
(
number INT
)
INSERT
INTO #tvar
SELECT number
FROM simple_intlist_to_tbl(#var)
SELECT *
FROM xyz, ... /* Other complex joins */
WHERE col1 IN
(
SELECT number
FROM #tvar
)
You need a function which splits your list into a data set (use this from Jeff Moden it works and is very fast) and then just use the IN clause on your desired column(s).
I have a table like this:
SKU Size
A 10
B 10
C 10
D 10
E 10
F 10
G 10
I want to change it to:
SKU Size
A 20
B 10
C 30
D 10
E 80
F 10
G 60
I have more than 3000 rows of records to update. How can I do that with SQL update command ?
UPDATE T
SET Size = CASE SKU
WHEN 'A' THEN 20
WHEN 'B' THEN 10
WHEN 'C' THEN 30
WHEN ...
END
Or there may be a formula for calculating the size, but you've failed to give it in your question (Or we may have to switch to a more complex CASE expression, but again, too little detail in the question).
Create a table with the mapping of SKU to new size; update the master table from that.
Many dialects of SQL have a notation for doing updates via joined tables. Some do not. This will work where there is no such notation:
CREATE TABLE SKU_Size_Map
(
SKU CHAR(16) NOT NULL,
Size INTEGER NOT NULL
);
...Populate this table with the SKU values to be set...
...You must have such a list...
UPDATE MasterTable
SET Size = (SELECT Size FROM SKU_Size_Map
WHERE MasterTable.SKU = SKU_Size_Map.Size)
WHERE SKU IN (SELECT SKU FROM SKU_Size_Map);
The main WHERE condition is need to avoid setting the size to null where there is no matching row.
You can probably also do it with a MERGE statement. But the key insight for any of these notations is that you need a table to do the mapping between SKU and size. You either need a table or you need an algorithm, and the sample data doesn't suggest an algorithm.
Make use of OpenXML to resolve your issue
example
declare #i int
exec sp_xml_preparedocument #i output,
'<mydata>
<test xmlID="3" xmlData="blah blah blah"/>
<test xmlID="1" xmlData="blah"/>
</mydata>'
insert into test
select xmlID, xmlData
from OpenXml(#i, 'mydata/test')
with (xmlID int, xmlData nvarchar(30))
where xmlID not in (select xmlID from test)
update test
set test.xmlData = ox.xmlData
from OpenXml(#i, 'mydata/test')
with (xmlID int, xmlData nvarchar(30)) ox
where test.xmlID = ox.xmlID
exec sp_xml_removedocument #i
Just do...
UPDATE [yourTable] SET Size = 20 WHERE SKU = 'A'
And do this for all values you want to change...
Well, if you don't have a formula to calculate your Sizes, and you don't have a file or an Excel sheet with the data that you can massage into your table, you'll just have to get some luckless intern to type something like
UPDATE <table> SET Size = <value> WHERE SKU = '<key>'
3000 times.
If you are that intern, I'd suggest giving us a little more information...
Since you wanted to change the whole column, drop that particular column by using this:
ALTER TABLE table_name
DROP COLUMN column_name;
then create a new column using:
ALTER TABLE table_name
ADD column_name varchar(80);
Scenario
I have a stored procedure written in T-Sql using SQL Server 2005.
"SEL_ValuesByAssetName"
It accepts a unique string "AssetName".
It returns a table of values.
Question
Instead of calling the stored procedure multiple times and having to make a database call everytime I do this, I want to create another stored procedure that accepts a list of all the "AssetNames", and calls the stored procedure "SEL_ValueByAssetName" for each assetname in the list, and then returns the ENTIRE TABLE OF VALUES.
Pseudo Code
foreach(value in #AllAssetsList)
{
#AssetName = value
SEL_ValueByAssetName(#AssetName)
UPDATE #TempTable
}
How would I go about doing this?
It will look quite crippled with using Stored Procedures. But can you use Table-Valued Functions instead?
In case of Table-Valued functions it would look something like:
SELECT al.Value AS AssetName, av.* FROM #AllAssetsList AS al
CROSS APPLY SEL_ValuesByAssetName(al.Value) AS av
Sample implementation:
First of all, we need to create a Table-Valued Parameter type:
CREATE TYPE [dbo].[tvpStringTable] AS TABLE(Value varchar(max) NOT NULL)
Then, we need a function to get a value of a specific asset:
CREATE FUNCTION [dbo].[tvfGetAssetValue]
(
#assetName varchar(max)
)
RETURNS TABLE
AS
RETURN
(
-- Add the SELECT statement with parameter references here
SELECT 0 AS AssetValue
UNION
SELECT 5 AS AssetValue
UNION
SELECT 7 AS AssetValue
)
Next, a function to return a list AssetName, AssetValue for assets list:
CREATE FUNCTION [dbo].[tvfGetAllAssets]
(
#assetsList tvpStringTable READONLY
)
RETURNS TABLE
AS
RETURN
(
-- Add the SELECT statement with parameter references here
SELECT al.Value AS AssetName, av.AssetValue FROM #assetsList al
CROSS APPLY tvfGetAssetValue(al.Value) AS av
)
Finally, we can test it:
DECLARE #names tvpStringTable
INSERT INTO #names VALUES ('name1'), ('name2'), ('name3')
SELECT * FROM [Test].[dbo].[tvfGetAllAssets] (#names)
In MSSQL 2000 I would make #allAssetsList a Varchar comma separated values list. (and keep in mind that maximum length is 8000)
I would create a temporary table in the memory, parse this string and insert into that table, then do a simple query with the condition where assetName in (select assetName from #tempTable)
I wrote about MSSQL 2000 because I am not sure whether MSSQL 2005 has some new data type like an array that can be passed as a literal to the SP.