SQL insert with regexp - sql

Question

I have a table with data in the following format
col1 col2
a1 a2;a3;a4
b1 b2
c1 c2;c3
d1 null
...
I'm trying to split the strings, get unique combinations of col1/col2 and insert them into tableB. So the expected outcome should look like this:
a1 a2
a1 a3
a1 a4
b1 b2
c1 c2
c1 c3
...
I tried the following query:
INSERT INTO tableB (col1, col2)
SELECT col1, (regexp_substr(col2,'[^;]+', 1, LEVEL)) FROM tableA
CONNECT BY regexp_substr(col2, '[^;]+', 1, LEVEL) IS NOT NULL;
Not sure what's going wrong here but it keeps executing (it actually went on for more than an hour) and when I finally cancel the task, nothing's been inserted. The table is quite large (around 25000 rows) but I've done similar inserts with larger tables and they worked fine.
I also tried adding a where clause (although it seems redundant) with
WHERE col2 LIKE'%;%'
That didn't help either.
Any suggestions would be great.
Edit: I tried counting the max number of substrings in col2, to ballpark the number of rows to be inserted, and found the max to be 42 substrings. The whole table has 25814 rows, so worst case scenario, it's inserting 1084104 rows. If that has anything to do with it.

Answer 1

Don't use connect by to split string into rows.
Use a PL/SQL procedure that does varchar2 -> collection split.
For a ad-hoc kind of query, stick with xmltable as a simple way to split string into rows (it is a bit slower than PL/SQL).
The following kind of query is expected to take 3-4 seconds for each input 1000 rows.
select t.col1, c2.val
from (
select 'a1' col1, 'a2;a3;a4' col2 from dual union all
select 'b1', 'b2' from dual union all
select 'c1', 'c2;c3' from dual union all
select 'd1', null from dual
) t
, xmltable('$WTF' passing
xmlquery(('"'||replace(replace(t.col2,'"','""'),';','","')||'"')
returning sequence
) as wtf
columns val varchar2(4000) path '.'
)(+) c2
Fiddle: http://sqlfiddle.com/#!4/9eecb7d/5059

Answer 2

One thing you can try is to select all the distinct values.
INSERT INTO tableB (col1, col2)
SELECT distinct col1, (regexp_substr(col2,'[^;]+', 1, LEVEL))
FROM tableA
CONNECT BY regexp_substr(col2, '[^;]+', 1, LEVEL) IS NOT NULL;
commit;
You should also commit the transaction if you need the changes to be permanent.

Answer 3

While I still don't really understand what was wrong with the initial query, I found a work around. The column with the strings I'm trying to split is participantcountries and the p_id contains unique identifiers.
CREATE OR REPLACE PROCEDURE split_countries IS
CURSOR get_rows IS
SELECT p_id, participantcountries
FROM staging_projects;
row_rec get_rows%ROWTYPE;
nb_countries number:=0;
country VARCHAR2(4);
BEGIN
OPEN get_rows;
LOOP
FETCH get_rows INTO row_rec;
EXIT WHEN get_rows%NOTFOUND;
SELECT regexp_count(row_rec.participantcountries, ';') INTO nb_countries
FROM staging_projects
WHERE p_id=row_rec.p_id;
IF (row_rec.participantcountries IS NULL) THEN
nb_countries:=0; -- if the field is null set counter to zero
ELSE
nb_countries:=nb_countries+1; -- nb of delimiters so +1 --> number of items
END IF;
WHILE (nb_countries >0) LOOP -- loop and get single country at current counter position
SELECT (regexp_substr(PARTICIPANTCOUNTRIES,'[^;]+', 1, nb_countries)) INTO country
FROM (SELECT * FROM staging_projects WHERE p_id=row_rec.p_id)
WHERE participantcountries IS NOT NULL;
nb_countries:=nb_countries-1;
INSERT INTO project_countries(proj_id, country_code) VALUES(row_rec.p_id, country);
END LOOP;
END LOOP;
CLOSE get_rows;
END split_countries;
This works fine but it seems like an overly complicated solution for the problem at hand.