SQL: break up one row into many (normalization) - sql

Question

I am in middle of upgrading from a poorly designed legacy database to a new database. In the old database there is tableA with fields Id and Commodities. Id is the primary key and contains an int and Commodities contains a comma delimited list.
TableA:
id | commodities
1135 | fish,eggs,meat
1127 | flour,oil
In the new database, I want tableB to be in the form id, commodity where each commodity is a single item from the comma delimited list in tableA.
TableB:
id | commodity
1135 | fish
1135 | eggs
1135 | meat
1127 | flour
1127 | oil
I have a function, functionA, that when given an id, a list, and a delimiter, returns a table with an id and item field. How can I use this function to turn the two fields from tableA into tableB?
(Note: I had trouble figuring out what to title this question. Please feel free to edit the title to make it more accurately reflect the question!)
Here is the function code:
ALTER FUNCTION dbo.functionA
(
#id int,
#List VARCHAR(6000),
#Delim varchar(5)
)
RETURNS
#ParsedList TABLE
(
id int,
item VARCHAR(6000)
)
AS
BEGIN
DECLARE #item VARCHAR(6000), #Pos INT
SET #List = LTRIM(RTRIM(#List))+ #Delim
SET #Pos = CHARINDEX(#Delim, #List, 1)
WHILE #Pos > 0
BEGIN
SET #item = LTRIM(RTRIM(LEFT(#List, #Pos - 1)))
IF #item <> ''
BEGIN
INSERT INTO #ParsedList (id, item)
VALUES (#id, CAST(#item AS VARCHAR(6000)))
END
SET #List = RIGHT(#List, LEN(#List) - #Pos)
SET #Pos = CHARINDEX(#Delim, #List, 1)
END
RETURN
END

Answer 1

Here's the link I posted as a comment:
http://www.sqlteam.com/article/parsing-csv-values-into-multiple-rows

Answer 2

You need a way to split and process the string in TSQL, there are many ways to do this. This article covers the PROs and CONs of just about every method:
Arrays and Lists in SQL Server 2000 and Earlier
You need to create a split function. This is how a split function can be used:
SELECT
*
FROM YourTable y
INNER JOIN dbo.yourSplitFunction(#Parameter) s ON y.ID=s.Value
[I prefer the number table approach to split a string in TSQL](Arrays and Lists in SQL Server 2000 and Earlier) but there are numerous ways to split strings in SQL Server, see the previous link, which explains the PROs and CONs of each.
For the Numbers Table method to work, you need to do this one time table setup, which will create a table Numbers that contains rows from 1 to 10,000:
SELECT TOP 10000 IDENTITY(int,1,1) AS Number
INTO Numbers
FROM sys.objects s1
CROSS JOIN sys.objects s2
ALTER TABLE Numbers ADD CONSTRAINT PK_Numbers PRIMARY KEY CLUSTERED (Number)
Once the Numbers table is set up, create this split function:
CREATE FUNCTION inline_split_me (#SplitOn char(1),#param varchar(7998)) RETURNS TABLE AS
RETURN(SELECT substring(#SplitOn + #param + ',', Number + 1,
charindex(#SplitOn, #SplitOn + #param + #SplitOn, Number + 1) - Number - 1)
AS Value
FROM Numbers
WHERE Number <= len(#SplitOn + #param + #SplitOn) - 1
AND substring(#SplitOn + #param + #SplitOn, Number, 1) = #SplitOn)
GO
You can now easily split a CSV string into a table and join on it:
select * from dbo.inline_split_me(';','1;22;333;4444;;') where LEN(Value)>0
OUTPUT:
Value
----------------------
1
22
333
4444
(4 row(s) affected)
to make you new table use this:
--set up tables:
create table TableA (id int, commodities varchar(8000))
INSERT TableA VALUES (1135,'fish,eggs,meat')
INSERT TableA VALUES (1127,'flour,oil')
Create table TableB (id int, commodities varchar(8000))
--populate TableB
INSERT TableB
(id, commodities)
SELECT
a.id,c.value
FROM TableA a
CROSS APPLY dbo.inline_split_me(',',a.commodities) c
--show tableB contents:
select * from TableB
OUTPUT:
id commodities
----------- -------------
1135 fish
1135 eggs
1135 meat
1127 flour
1127 oil
(5 row(s) affected)
EDIT after Conrad Frix comment about SQL Server 2000 not supporting CROSS APPLY
this will do the same:
INSERT TableB
(id, commodities)
SELECT
a.id,NullIf(SubString(',' + a.commodities + ',' , number , CharIndex(',' , ',' + a.commodities + ',' , number) - number) , '')
FROM TableA a
INNER JOIN Numbers n ON 1=1
WHERE SubString(',' + a.commodities + ',' , number - 1, 1) = ','
AND CharIndex(',' , ',' + a.commodities + ',' , number) - number > 0
AND number <= Len(',' + a.commodities + ',')
and is based on the code from the link in the answer by #Rup. It basically removes the function call and does the split in the main query (using a similar Numbers table split), so no need for a CROSS APPLY.

Answer 3

You write a SQL batch that loops through table A and inserts into table b the results of your function call.

Answer 4

Call me lazy, but I'd pull the combined rows out of the database, split them, then reinsert the split rows. This kind of thing seems kind of unnatural for SQL...

Answer 5

SSIS has a pretty handy Unpivot transform if that's available to you.

Answer 6

create table Project (ProjectId int, Description varchar(50));
insert into Project values (1, 'Chase tail, change directions');
insert into Project values (2, 'ping-pong ball in clothes dryer');
create table ProjectResource (ProjectId int, ResourceId int, Name varchar(15));
insert into ProjectResource values (1, 1, 'Adam');
insert into ProjectResource values (1, 2, 'Kerry');
insert into ProjectResource values (1, 3, 'Tom');
insert into ProjectResource values (2, 4, 'David');
insert into ProjectResource values (2, 5, 'Jeff');
-- a bit of SQL magic involving XML and voila
SELECT *,
(SELECT Name + ' ' AS [text()]
FROM ProjectResource pr
WHERE pr.ProjectId = p.ProjectId
FOR XML PATH ('')) AS ResourceList
FROM Project p
The output of this will be :
ProjectId Description ResourceList
1 Chase tail, change directions Adam Kerry Tom
2 ping-pong ball in clothes dryer David Jeff