Related
Using Oracle 12c DB, I have the following table data example that I need assistance with using SQL and PL/SQL.
Table data is as follows:
Table Name: my_data
ID ITEM ITEM_LOC
------- ----------- ----------------
1 Item-1 0,1
2 Item-2 0,1,2,3,4,7
3 Item-3 0-48
4 Item-4 0,1,2,3,4,5,6,7,8
5 Item-5 1-33
6 Item-6 0,1
7 Item-7 0,1,5,8
Using the data above within the my_data table, what is the best way to process this ITEM_LOC as I need to use the values in this column as an individual value, i.e:
0,1 means the SQL needs to return either 0 or 1 or
range values, i.e:
0-48 means the SQL needs to return a value between 0 and 48.
The returned values for both scenarios should commence from lowest to highest and can't be re-used once processed.
Based on the above, it would be great to have a function that takes the ID and returns an individual value from ITEM_LOC that hasn't been used, based on my description above. This could be a comma-separated string value or a range string value.
Desired result for ID = 2 could be 7. For this ID = 2, ITEM_LOC = 7 could not be used again.
Desired result for ID = 5 could be 31. For this ID = 5, ITEM_LOC = 31 could not be used again.
For the ITEM_LOC data that could not be used again, against that ID, I am looking at holding another table to hold this or perhaps separate all data into separate rows with a new column called VALUE_USED.
This query shows how to extract list of ITEM_LOC values based on whether they are comma-separated (which means "take exactly those values") or dash-separated (which means "find all values between starting and end point"). I modified your sample data a little bit (didn't feel like displaying ~50 values if 5 of them do the job).
lines #1 - 6 represent sample data.
the first select (lines #7 - 15) splits comma-separated values into rows
the second select (lines #17 - 26) uses a hierarchical query which adds 1 to the starting value, up to item's end value.
SQL> with my_data (id, item, item_loc) as
2 (select 2, 'Item-2', '0,2,4,7' from dual union all
3 select 7, 'Item-7', '0,1,5' from dual union all
4 select 3, 'Item-3', '0-4' from dual union all
5 select 8, 'Item-8', '5-8' from dual
6 )
7 select id,
8 item,
9 regexp_substr(item_loc, '[^,]+', 1, column_value) loc
10 from my_data
11 cross join table(cast(multiset
12 (select level from dual
13 connect by level <= regexp_count(item_loc, ',') + 1
14 ) as sys.odcinumberlist))
15 where instr(item_loc, '-') = 0
16 union all
17 select id,
18 item,
19 to_char(to_number(regexp_substr(item_loc, '^\d+')) + column_value - 1) loc
20 from my_data
21 cross join table(cast(multiset
22 (select level from dual
23 connect by level <= to_number(regexp_substr(item_loc, '\d+$')) -
24 to_number(regexp_substr(item_loc, '^\d+')) + 1
25 ) as sys.odcinumberlist))
26 where instr(item_loc, '-') > 0
27 order by id, item, loc;
ID ITEM LOC
---------- ------ ----------------------------------------
2 Item-2 0
2 Item-2 2
2 Item-2 4
2 Item-2 7
3 Item-3 0
3 Item-3 1
3 Item-3 2
3 Item-3 3
3 Item-3 4
7 Item-7 0
7 Item-7 1
7 Item-7 5
8 Item-8 5
8 Item-8 6
8 Item-8 7
8 Item-8 8
16 rows selected.
SQL>
I don't know what you meant by saying that "item_loc could not be used again". Used where? If you use the above query in, for example, cursor FOR loop, then yes - those values would be used only once as every loop iteration fetches next item_loc value.
As others have said, it's a bad idea to store data in this way. You very likely could have input like this, and you likely could need to display the data like this, but you don't have to store the data the way it is input or displayed.
I'm going to store the data as individual LOC elements based on the input. I assume the data contains only integers separated by commas, or pairs of integers separated by a hyphen. Whitespace is ignored. The comma-separated list does not have to be in any order. In pairs, if the left integer is greater than the right integer I return no LOC element.
create table t as
with input(id, item, item_loc) as (
select 1, 'Item-1', ' 0,1' from dual union all
select 2, 'Item-2', '0,1,2,3,4,7' from dual union all
select 3, 'Item-3', '0-48' from dual union all
select 4, 'Item-4', '0,1,2,3,4,5,6,7,8' from dual union all
select 5, 'Item-5', '1-33' from dual union all
select 6, 'Item-6', '0,1' from dual union all
select 7, 'Item-7', '0,1,5,8,7 - 11' from dual
)
select distinct id, item, loc from input, xmltable(
'let $item := if (contains($X,",")) then ora:tokenize($X,"\,") else $X
for $i in $item
let $j := if (contains($i,"-")) then ora:tokenize($i,"\-") else $i
for $k in xs:int($j[1]) to xs:int($j[count($j)])
return $k'
passing item_loc as X
columns loc number path '.'
);
Now to "use" an element I just delete it from the table:
delete from t where rowid = (
select min(rowid) keep (dense_rank first order by loc)
from t
where id = 7
);
To return the data in the same format it was input, use MATCH_RECOGNIZE:
select id, item, listagg(item_loc, ',') within group(order by first_loc) item_loc
from t
match_recognize(
partition by id, item order by loc
measures a.loc first_loc,
a.loc || case count(*) when 1 then null else '-'||b.loc end item_loc
pattern (a b*)
define b as loc = prev(loc) + 1
)
group by id, item;
ID ITEM ITEM_LOC
1 Item-1 0-1
2 Item-2 0-4,7
3 Item-3 0-48
4 Item-4 0-8
5 Item-5 1-33
6 Item-6 0-1
7 Item-7 1,5,7-11
Note that the output here will not be exactly like the input, because any consecutive integers will be compressed into a pair.
I took a look into the forums and couldn't really find something that I needed.
What I have is two tables one table with (Parse_Table)
File_ID|Start_Pos|Length|Description
------------------------------------
1 | 1 | 9 | Pos1
1 | 10 | 1 | Pos2
1 | 11 | 1 | Pos3
2 | 1 | 4 | Pos1
2 | 5 | 7 | Pos2
and another table that needs to be parsed like (Input_file)
String
ABCDEFGHI12
ASRQWERTQ45
123456789AB
321654852PO
and I want to have the result where If I put it will use this specific parsing spec
select DESCRIPTION, Start_pos,Length from Parse_table where File_ID=1
and be able to parse input file
String | Pos1 |Pos2|Pos3
---------------------------------
ABCDEFGHI12 |ABCDEFGHI | 1 | 2
ASRQWERTQ45 |ASRQWERTQ | 4 | 5
123456789AB |123456789 | A | B
321654852PO |321654852 | P | O
and alternatively if I put file_id=2 it would parse the values differently.
I looked at using the Pivot function, but it looks like number of columns are static, at least to my knowledge.
thanks in advance for your support please let me know what I can do in SQL.
You can get "close-ish" with the standard decode tricks to pivot the table assuming a ceiling on the maximum number of fields expected.
SQL> create table t ( fid int, st int, len int, pos varchar2(10));
Table created.
SQL>
SQL> insert into t values ( 1 , 1 , 9 , 'Pos1');
1 row created.
SQL> insert into t values ( 1 , 10 , 1 , 'Pos2');
1 row created.
SQL> insert into t values ( 1 , 11 , 1 , 'Pos3');
1 row created.
SQL> insert into t values ( 2 , 1 , 4 , 'Pos1');
1 row created.
SQL> insert into t values ( 2 , 5 , 7 , 'Pos2');
1 row created.
SQL>
SQL> create table t1 ( s varchar2(20));
Table created.
SQL>
SQL> insert into t1 values ('ABCDEFGHI12');
1 row created.
SQL> insert into t1 values ('ASRQWERTQ45');
1 row created.
SQL> insert into t1 values ('123456789AB');
1 row created.
SQL> insert into t1 values ('321654852PO');
1 row created.
SQL>
SQL>
SQL> select
2 t1.s,
3 max(decode(t.seq,1,substr(t1.s,t.st,t.len))) c1,
4 max(decode(t.seq,2,substr(t1.s,t.st,t.len))) c2,
5 max(decode(t.seq,3,substr(t1.s,t.st,t.len))) c3,
6 max(decode(t.seq,4,substr(t1.s,t.st,t.len))) c4,
7 max(decode(t.seq,5,substr(t1.s,t.st,t.len))) c5,
8 max(decode(t.seq,6,substr(t1.s,t.st,t.len))) c6
9 from t1,
10 ( select t.*, row_number() over ( partition by fid order by st ) as seq
11 from t
12 where fid = 1
13 ) t
14 group by t1.s
15 order by 1;
S C1 C2 C3 C4 C5 C6
-------------------- ------------- ------------- ------------- ------------- ------------- -------------
123456789AB 123456789 A B
321654852PO 321654852 P O
ABCDEFGHI12 ABCDEFGHI 1 2
ASRQWERTQ45 ASRQWERTQ 4 5
4 rows selected.
SQL>
SQL> select
2 t1.s,
3 max(decode(t.seq,1,substr(t1.s,t.st,t.len))) c1,
4 max(decode(t.seq,2,substr(t1.s,t.st,t.len))) c2,
5 max(decode(t.seq,3,substr(t1.s,t.st,t.len))) c3,
6 max(decode(t.seq,4,substr(t1.s,t.st,t.len))) c4,
7 max(decode(t.seq,5,substr(t1.s,t.st,t.len))) c5,
8 max(decode(t.seq,6,substr(t1.s,t.st,t.len))) c6
9 from t1,
10 ( select t.*, row_number() over ( partition by fid order by st ) as seq
11 from t
12 where fid = 2
13 ) t
14 group by t1.s
15 order by 1;
S C1 C2 C3 C4 C5 C6
-------------------- ------------- ------------- ------------- ------------- ------------- -------------
123456789AB 1234 56789AB
321654852PO 3216 54852PO
ABCDEFGHI12 ABCD EFGHI12
ASRQWERTQ45 ASRQ WERTQ45
4 rows selected.
If you really wanted that result to then come back with only the desired column count and custom column names, then you're into dynamic SQL territory. How you'd tackle that depends on the tool you are providing the data to. If it can consume a REF CURSOR, then a little PL/SQL would do the trick.
An unknown number of columns can be returned from a SQL statement, but it requires code built with PL/SQL, ANY types, and Oracle Data Cartridge.
That code is tricky to write but you can start with my open source project Method4. Download, unzip, #install, and then
write a SQL statement to generate a SQL statement.
Query
select * from table(method4.dynamic_query(
q'[
--Create a SQL statement to query PARSE_FILE.
select
'select '||
listagg(column_expression, ',') within group (order by start_pos) ||
' from parse_file'
column_expressions
from
(
--Create individual SUBSTR column expressions.
select
parse_table.*,
'substr(string, '||start_pos||', '||length||') '||description column_expression
from parse_table
--CHANGE BELOW LINE TO USE A DIFFERENT FILE:
where file_id = 2
order by start_pos
)
]'
));
Sample Schema
create table parse_table as
select 1 file_id, 1 start_pos, 9 length, 'Pos1' description from dual union all
select 1 file_id, 10 start_pos, 1 length, 'Pos2' description from dual union all
select 1 file_id, 11 start_pos, 1 length, 'Pos3' description from dual union all
select 2 file_id, 1 start_pos, 4 length, 'Pos1' description from dual union all
select 2 file_id, 5 start_pos, 7 length, 'Pos2' description from dual;
create table parse_file as
select 'ABCDEFGHI12' string from dual union all
select 'ASRQWERTQ45' string from dual union all
select '123456789AB' string from dual union all
select '321654852PO' string from dual;
Results
When FILE_ID = 1:
POS1 POS2 POS3
---- ---- ----
ABCDEFGHI 1 2
ASRQWERTQ 4 5
123456789 A B
321654852 P O
When FILE_ID = 2:
POS1 POS2
---- ----
ABCD EFGHI12
ASRQ WERTQ45
1234 56789AB
3216 54852PO
The motivation for this challenge was to easily and accurately simulate a data set of IP ranges that relate to each other in a certain way.
The Challenge
A table contains a single column of text type.
The text contains one or more lines where each lines contains one or more sections created from dashes.
The goal is to write a query that returns a tuple for each section with its start point and end point.
E.g.
'
--- -- -
----
'
The text above contains 2 lines.
It contains 4 sections.
The 1st line contains 3 sections.
The 2nd line contains 1 section.
The tuples for the 1st lines are (1,3),(5,6),(8,8).
The tuple for the 2nd line is (2,5).
Requirements
The solution should be a single SQL query (sub-queries are fine).
The use of T-SQL, PL/SQL etc. is not allowed.
The use of UDF (User Defined Functions) is not allowed
If needed, we might assume that there is only a single record in the table.
Sample Data
create table t (txt varchar (1000) not null);
insert into t (txt) values
(
'
--- --- --- ---
---------- -
- - -- -- --- ---
----- ---- --- -- -
-------
'
);
Requested Result
* Only the last 2 columns (section_start/end) are required, the rest are for debugging purposes.
line_ind section_ind section_length section_start section_end
-------- ----------- -------------- ------------- -----------
1 1 3 2 4
1 2 3 6 8
1 3 3 11 13
1 4 3 17 19
2 1 10 1 10
2 2 1 21 21
3 1 1 2 2
3 2 1 4 4
3 3 2 6 7
3 4 2 9 10
3 5 3 12 14
3 6 3 16 18
4 1 5 7 11
4 2 4 13 16
4 3 3 18 20
4 4 2 22 23
4 5 1 25 25
5 1 7 4 10
Oracle
SELECT row_n AS line_ind
,dense_rank() over(PARTITION BY row_n ORDER BY s_beg) AS section_ind
,s_end - s_beg AS section_length
,s_beg - decode(row_n, 0, 0, instr(a,chr(10),1,row_n)) AS section_start
,s_end - decode(row_n, 0, 0, instr(a,chr(10),1,row_n)) -1 AS section_end
FROM (SELECT a
,s_beg
,DECODE(s_end, 0, length(a) + 1, s_end) AS s_end
,length(substr(a, 1, s_beg))
- length(REPLACE(substr(a, 1, s_beg), chr(10))) AS row_n
,lvl
FROM (SELECT txt as a
,DECODE(LEVEL, 1, 0, regexp_instr(txt , '\s|\n', 1, LEVEL - 1)) + 1 AS s_beg
,regexp_instr(txt , '\s|\n', 1, LEVEL) AS s_end
,LEVEL AS lvl
FROM t
CONNECT BY LEVEL <= length(txt ) - length(regexp_replace(txt , '\s|\n')) + 1)
)WHERE s_beg != s_end;
Oracle
select regexp_instr (txt,'-+',1,level,0) - instr (txt,chr(10),regexp_instr (txt,'-+',1,level,0) - length (txt) - 1,1) as section_start
,regexp_instr (txt,'-+',1,level,1) - 1 - instr (txt,chr(10),regexp_instr (txt,'-+',1,level,0) - length (txt) - 1,1) as section_end
from t
connect by level <= regexp_count (txt,'-+')
;
Teradata
with l
as
(
select line_ind
,line
from table
(
regexp_split_to_table (-1,t.txt,'\r','')
returns (minus_one int,line_ind int,line varchar(1000))
)
as l
)
select l.line_ind
,s.section_ind
,regexp_instr (l.line,'\S+',1,s.section_ind,0) as section_start
,regexp_instr (l.line,'\S+',1,s.section_ind,1) - 1 as section_end
,char_length (s.section) as section_length
from table
(
regexp_split_to_table (l.line_ind,l.line,'\s+','')
returns (line_ind int,section_ind int,section varchar(1000))
)
as s
,l
where l.line_ind =
s.line_ind
order by l.line_ind
,s.section_ind
;
I have created a query that will output a flat file with header and details.
Now, I want to add a trailer record that will contain the total count of the detail records.
I have correctly counted the total of records using row_number, but it displays the every record.
How can I get the last line so that it will reflect the total count in the trailer line.
This is the code I already created for the headers and detail.
SQL> SELECT filerec FROM (
2 SELECT 'FILENAME' AS filerec, 1 col FROM dual
3 UNION ALL
4 SELECT 'FILEDATE: ' || to_char(SYSDATE,'mm/dd/yyyy') as filerec, 2 col FROM dual
5 UNION ALL
6 SELECT empno || ename AS filerec, NULL col FROM emp
7 ORDER BY 2,1
8 );
This is the output I want to get. (added the last rec, 'TRAILER: 0004')
FILENAME
FILEDATE: 02/27/2015
7369SMITH
7499ALLEN
7521WARD
7566JONES
TRAILER: 0004
There are several ways to do this. I'd prefer to use grouping. See how it can be used. Let we have a recordset:
SQL> select
2 'Row ' || rownum
3 from
4 dual
5 connect by
6 level <= 5;
'ROW'||ROWNUM
--------------------------------------------
Row 1
Row 2
Row 3
Row 4
Row 5
Now we wish to add counting:
SQL> select
2 case
3 when grouping_id(rownum) = 0 then 'Row ' || rownum
4 else 'Total: ' || count(*) || ' row(s)'
5 end
6 from
7 dual
8 connect by
9 level <= 5
10 group by rollup (rownum);
CASEWHENGROUPING_ID(ROWNUM)=0T
------------------------------------------------------
Row 1
Row 2
Row 3
Row 4
Row 5
Total: 5 row(s)
6 rows selected
I have de-normalized table, something like
CODES
ID | VALUE
10 | A,B,C
11 | A,B
12 | A,B,C,D,E,F
13 | R,T,D,W,W,W,W,W,S,S
The job is to convert is where each token from VALUE will generate new row. Example:
CODES_TRANS
ID | VALUE_TRANS
10 | A
10 | B
10 | C
11 | A
11 | B
What is the best way to do it in PL/SQL without usage of custom pl/sql packages, ideally with pure SQL?
Obvious solution is to implement it via cursors. Any ideas?
Another alternative is to use the model clause:
SQL> select id
2 , value
3 from codes
4 model
5 return updated rows
6 partition by (id)
7 dimension by (-1 i)
8 measures (value)
9 ( value[for i from 0 to length(value[-1])-length(replace(value[-1],',')) increment 1]
10 = regexp_substr(value[-1],'[^,]+',1,cv(i)+1)
11 )
12 order by id
13 , i
14 /
ID VALUE
---------- -------------------
10 A
10 B
10 C
11 A
11 B
12 A
12 B
12 C
12 D
12 E
12 F
13 R
13 T
13 D
13 W
13 W
13 W
13 W
13 W
13 S
13 S
21 rows selected.
I have written up to 6 alternatives for this type of query in this blogpost: http://rwijk.blogspot.com/2007/11/interval-based-row-generation.html
Regards,
Rob.
I have a pure SQL solution for you.
I adapted a trick I found on an old Ask Tom site, posted by Mihail Bratu. My adaptation uses regex to tokenise the VALUE column, so it requires 10g or higher.
The test data.
SQL> select * from t34
2 /
ID VALUE
---------- -------------------------
10 A,B,C
11 A,B
12 A,B,C,D,E,F
13 R,T,D,W1,W2,W3,W4,W5,S,S
SQL>
The query:
SQL> select t34.id
2 , t.column_value value
3 from t34
4 , table(cast(multiset(
5 select regexp_substr (t34.value, '[^(,)]+', 1, level)
6 from dual
7 connect by level <= length(value)
8 ) as sys.dbms_debug_vc2coll )) t
9 where t.column_value != ','
10 /
ID VALUE
---------- -------------------------
10 A
10 B
10 C
11 A
11 B
12 A
12 B
12 C
12 D
12 E
12 F
13 R
13 T
13 D
13 W1
13 W2
13 W3
13 W4
13 W5
13 S
13 S
21 rows selected.
SQL>
Based on Celko's book, here is what I found and it's working well!
SELECT
TABLE1.ID
, MAX(SEQ1.SEQ) AS START_POS
, SEQ2.SEQ AS END_POS
, COUNT(SEQ2.SEQ) AS PLACE
FROM
TABLE1, V_SEQ SEQ1, V_SEQ SEQ2
WHERE
SUBSTR(',' || TABLE1.VALUE || ',', SEQ1.SEQ, 1) = ','
AND SUBSTR(',' || TABLE1.VALUE || ',', SEQ2.SEQ, 1) = ','
AND SEQ1.SEQ < SEQ2.SEQ
AND SEQ2.SEQ <= LENGTH(TABLE1.VALUE)
GROUP BY TABLE1.ID, TABLE1.VALUE, SEQ2.SEQ
Where V_SEQ is a static table with one field:
SEQ, integer values 1 through N, where N >= MAX_LENGTH(VALUE).
This is based on the fact the the VALUE is wrapped by ',' on both ends, like this:
,A,B,C,D,
If your tokens are fixed length (like in my case) I simply used PLACE field to calculate the actual string. If variable length, use start_pos and end_pos
So, in my case, tokens are 2 char long, so the final SQL is:
SELECT
TABLE1.ID
, SUBSTR(TABLE1.VALUE, T_SUB.PLACE * 3 - 2 , 2 ) AS SINGLE_VAL
FROM
(
SELECT
TABLE1.ID
, MAX(SEQ1.SEQ) AS START_POS
, SEQ2.SEQ AS END_POS
, COUNT(SEQ2.SEQ) AS PLACE
FROM
TABLE1, V_SEQ SEQ1, V_SEQ SEQ2
WHERE
SUBSTR(',' || TABLE1.VALUE || ',', SEQ1.SEQ, 1) = ','
AND SUBSTR(',' || TABLE1.VALUE || ',', SEQ2.SEQ, 1) = ','
AND SEQ1.SEQ < SEQ2.SEQ
AND SEQ2.SEQ <= LENGTH(TABLE1.VALUE)
GROUP BY TABLE1.ID, TABLE1.VALUE, SEQ2.SEQ
) T_SUB
INNER JOIN
TABLE1 ON TABLE1.ID = T_SUB.ID
ORDER BY TABLE1.ID, T_SUB.PLACE
Original Answer
In SQL Server TSQL we parse strings and make a table object. Here is sample code - maybe you can translate it.
http://rbgupta.blogspot.com/2007/10/tsql-parsing-delimited-string-into.html
Second Option
Count the number of commas per row. Get the Max number of commas. Let's say that in the entire table you have a row with 5 commas max. Build a SELECT with 5 substrings. This will make it a set based operation and should be much faster than a rbar.