Related
i am using oracle sql. i would like to substr starting from characters XY0 and include 2 or 3 more characters until '-' sign in the string
These characters may be anywhere in the string.
Original
column_value
1st Row - Error due to XY0066- Does not fit -Not suitable
2nd Row -Error due to specific XY0089- Will not match
3rd Row -Not in good cond XY0215- Special type error
Extraction should be
result
XY0066
XY0089
XY0215
How can I do this?
You can use:
SELECT id,
SUBSTR(value, start_pos, end_pos - start_pos) AS code
FROM (
SELECT id,
value,
INSTR(value, 'XY') AS start_pos,
INSTR(value, '-', INSTR(value, 'XY') + 2) AS end_pos
FROM table_name
);
or
SELECT id,
SUBSTR(
value,
INSTR(value, 'XY'),
INSTR(value, '-', INSTR(value, 'XY') + 2) - INSTR(value, 'XY')
) AS code
FROM table_name;
or using regular expressions, which is shorter to type but will run much slower:
SELECT id,
REGEXP_SUBSTR(value, 'XY[^-]*') AS code
FROM table_name;
Which, for the sample data:
CREATE TABLE table_name (id, value) AS
SELECT 1, 'Error due to XY0066- Does not fit -Not suitable' FROM DUAL UNION ALL
SELECT 2, 'Error due to specific XY0089- Will not match' FROM DUAL UNION ALL
SELECT 3, 'Not in good cond XY0215- Special type error' FROM DUAL;
All output:
ID
CODE
1
XY0066
2
XY0089
3
XY0215
fiddle
Hil All,
I have a table , count is about 200M. It has a column which contains data separated by '~'. I want to parse it.
e.g:
Column1
A~B~C~D~E~F
Result :
Column_new1
A~C~E
I just want to skip 2,4,6,n th. words. I don't want plsql. I need sql query. And table is very big,I also need performance.
I use substr,instr functions and I can parse. But it runs really slowly..
Thanks for help.
If you are after performance then use the INSTR and SUBSTR simple string functions:
SELECT SUBSTR(column1, 1, p1 - 1 ) || '~' ||
SUBSTR(column1, p2 + 1, p3 - p2 - 1) || '~' ||
SUBSTR(column1, p4 + 1, p5 - p4 - 1) AS column1_new
FROM (
SELECT column1,
INSTR(column1, '~', 1, 1) AS p1,
INSTR(column1, '~', 1, 2) AS p2,
INSTR(column1, '~', 1, 3) AS p3,
INSTR(column1, '~', 1, 4) AS p4,
INSTR(column1, '~', 1, 5) AS p5
FROM table_name
);
Which, for the sample data:
CREATE TABLE table_name (column1) AS
SELECT 'A~B~C~D~E~F' FROM DUAL;
Outputs:
COLUMN1_NEW
A~C~E
If you want a shorter query then you can use regular expressions:
SELECT REGEXP_REPLACE(column1, '([^~]+)~[^~]+~([^~]+)~[^~]+~([^~]+).*', '\1~\2~\3' )
AS column1_new
FROM table_name;
However, you will find that performance is likely to be an order of magnitude worse than simple string functions.
Another alternative would be to generate a materialized view.
db<>fiddle here
This is a regular expressions option. Looks nice, isn't PL/SQL, works OK (for 2 rows). I'm afraid that anything will run slow for 200 million rows.
SQL> with test (id, col) as
2 (select 1, 'A~B~C~D~E~F' from dual union all
3 select 2, 'M~N~O~P~Q-R' from dual
4 )
5 select id,
6 regexp_substr(col, '\w+', 1, 1) || '~' ||
7 regexp_substr(col, '\w+', 1, 3) || '~' ||
8 regexp_substr(col, '\w+', 1, 5) result
9 from test;
ID RESULT
---------- -----------------------------------
1 A~C~E
2 M~O~Q
SQL>
Hello folks & fellow developers,
I am working on an Oracle DB and have some XML data stored in blob format,
The XML has:
a) a formula (eg. %1 - %2 , or %2||'-'||%1)
b) a delimited list of variables that fit into the formula (eg. SALE_Q1| SALE_Q2 or YEAR| MONTH)
I have managed to pull this data into 2 different columns (can be merged into 1 column if needed too) and what I need to do is get the output column as a superimposition of the variables onto the placeholders.
(eg. SALE_Q1 - SALE_Q2 , or MONTH||'-'||YEAR)
there are also a few caveats to this like:
I don't know how many variables will each formula have and,
the variables will not always be used in the formula in the same order as the delimited list (see eg.2)
we can consider the data coming from a query like: SELECT formula || ', ' || columns_used FROM data_table; for the input string
and the output i am currently getting is like:
%1||'-'||%2||%3, SITE_NO| SITE_NAME| COUNTRY
0.1 * %1, WIND_RES
%1, TOTAL
CASE WHEN LENGTH(%1) < 8 THEN NULL ELSE TO_DATE(%1,'yyyymmdd')END, MIN_DATE
%1(+)=%3 and %2(+)=%4, ABC| LMN| PQR| XYZ
I am pretty new to SQL and this requirement is going over my head, any help will be highly appreciated. I need an SQL solution as a PL/SQL solution is not feasible in my requirement (this script is going to pull data from one DB and feed it to another repo on a regular basis)
I have seen some articles on XML table, model, or recursive regexp but I am not sure how I can use these. I have also seen this question on StackOverflow but my requirement is a little different from that and is a bit trickier as well. getting the formula string and the variables into the same string for processing like that question is also a plausible route. Any solution that can be written in an SQL query will be very helpful.
some more examples for your consideration:
"%1||'-'||%2||%3, SITE_NO| SITE_NAME| COUNTRY" => "SITE_NO||'-'||SITE_NAME||COUNTRY"
"0.1 * %1, WIND_RES" => "0.1 * WIND_RES"
"%1, TOTAL" => "TOTAL"
"CASE WHEN LENGTH(%1) < 8 THEN NULL ELSE TO_DATE(%1,'yyyymmdd')END, MIN_DATE" => "CASE WHEN LENGTH(MIN_DATE) < 8 THEN NULL ELSE TO_DATE(MIN_DATE,'yyyymmdd')END"
"%1(+)=%3 and %2(+)=%4, ABC| LMN| PQR| XYZ" => "ABC(+)=PQR and LMN(+)=XYZ"
I am assuming that the last comma is the delimiter between the template string and the delimited terms. You can use a recursive sub-query factoring clause and simple string functions:
WITH terms ( value, terms, num_terms ) AS (
SELECT SUBSTR( value, 1, INSTR( value, ', ', -1 ) - 1 ),
SUBSTR( value, INSTR( value, ', ', -1 ) + 2 ),
REGEXP_COUNT(
SUBSTR( value, INSTR( value, ', ', -1 ) + 2 ),
'.+?(\| |$)'
)
FROM table_name
),
term_bounds ( rn, value, terms, start_pos, lvl ) AS (
SELECT ROWNUM,
REPLACE(
value,
'%' || num_terms,
CASE num_terms
WHEN 1
THEN terms
ELSE SUBSTR( terms, INSTR( terms, '| ', 1, num_terms - 1 ) + 2 )
END
),
terms,
CASE
WHEN num_terms > 1
THEN INSTR( terms, '| ', 1, num_terms - 1 )
ELSE 1
END,
num_terms
FROM terms
UNION ALL
SELECT rn,
REPLACE(
value,
'%' || (lvl - 1),
CASE lvl - 1
WHEN 1
THEN SUBSTR( terms, 1, start_pos - 1 )
ELSE SUBSTR(
terms,
INSTR( terms, '| ', 1, lvl - 2 ) + 2,
start_pos - INSTR( terms, '| ', 1, lvl - 2 ) - 2
)
END
),
terms,
CASE
WHEN lvl > 2
THEN INSTR( terms, '| ', 1, lvl - 2 )
ELSE 1
END,
lvl - 1
FROM term_bounds
WHERE lvl > 1
)
SEARCH DEPTH FIRST BY rn SET rn_order
SELECT value
FROM term_bounds
WHERE lvl = 1;
Which, for the sample data:
CREATE TABLE table_name ( value ) AS
SELECT '%1||'-'||%2||%3, SITE_NO| SITE_NAME| COUNTRY' FROM DUAL UNION ALL
SELECT '0.1 * %1, WIND_RES' FROM DUAL UNION ALL
SELECT '%1, TOTAL' FROM DUAL UNION ALL
SELECT 'CASE WHEN LENGTH(%1) < 8 THEN NULL ELSE TO_DATE(%1,'yyyymmdd')END, MIN_DATE' FROM DUAL UNION ALL
SELECT '%1(+)=%3 and %2(+)=%4, ABC| LMN| PQR| XYZ' FROM DUAL UNION ALL
SELECT '%1, %2, %3, %4, %5, %6, %7, %8, %9, %10, %11, ONE| TWO| THREE| FOUR| FIVE| SIX| SEVEN| EIGHT| NINE| TEN| ELEVEN' FROM DUAL UNION ALL
SELECT '%%%%%%%7, HELLO| 1| 2| 3| 4| 5| 6' FROM DUAL
Outputs:
| VALUE |
| :----------------------------------------------------------------------------------------- |
| SITE_NO||'-'||SITE_NAME||COUNTRY |
| 0.1 * WIND_RES |
| TOTAL |
| CASE WHEN LENGTH(MIN_DATE) < 8 THEN NULL ELSE TO_DATE(MIN_DATE,'yyyymmdd')END |
| ABC(+)=PQR and LMN(+)=XYZ |
| ONE, TWO, THREE, FOUR, FIVE, SIX, SEVEN, EIGHT, NINE, TEN, ELEVEN |
| HELLO |
db<>fiddle here
Or as a PL/SQL function:
CREATE FUNCTION substitute_values(
i_value IN VARCHAR2,
i_terms IN VARCHAR2,
i_term_delimiter IN VARCHAR2 DEFAULT '| '
) RETURN VARCHAR2 DETERMINISTIC
IS
TYPE term_list_type IS TABLE OF VARCHAR2(200);
v_start PLS_INTEGER := 1;
v_end PLS_INTEGER;
v_index PLS_INTEGER;
v_terms term_list_type := term_list_type();
v_output VARCHAR2(4000) := i_value;
BEGIN
LOOP
v_end := INSTR(i_terms, i_term_delimiter, v_start);
v_terms.EXTEND;
IF v_end > 0 THEN
v_terms(v_terms.COUNT) := SUBSTR(i_terms, v_start, v_end - v_start);
ELSE
v_terms(v_terms.COUNT) := SUBSTR(i_terms, v_start);
EXIT;
END IF;
v_start := v_end + LENGTH(i_term_delimiter);
END LOOP;
LOOP
v_index := TO_NUMBER(REGEXP_SUBSTR(v_output, '^(.*?)%(\d+)', 1, 1, NULL, 2));
IF v_index IS NULL THEN
RETURN v_output;
ELSIF v_index > v_terms.COUNT THEN
RETURN NULL;
END IF;
v_output := REGEXP_REPLACE(v_output, '^(.*?)%(\d+)', '\1' || v_terms(v_index));
END LOOP;
END;
/
Then:
SELECT SUBSTITUTE_VALUES(
SUBSTR(value, 1, INSTR(value, ', ', -1) - 1),
SUBSTR(value, INSTR(value, ', ', -1) + 2)
) AS value
FROM table_name
Outputs:
VALUE
SITE_NO||'-'||SITE_NAME||COUNTRY
0.1 * WIND_RES
TOTAL
CASE WHEN LENGTH(MIN_DATE) < 8 THEN NULL ELSE TO_DATE(MIN_DATE,'yyyymmdd')END
ABC(+)=PQR and LMN(+)=XYZ
ONE, TWO, THREE, FOUR, FIVE, SIX, SEVEN, EIGHT, NINE, TEN, ELEVEN
HELLO
db<>fiddle here
I created a query and it's properly working.. but I'm not yet satisfied because my code is too long, is there way that I can simplify or shorten my select statement ?
select
/*GenInfo*/
id ,name name,
replace(regexp_substr(properties, 'EntityID=[^;]*'), 'EntityID=', '') as EntityID,
replace(regexp_substr(properties, 'deployed=[^;]*'), 'deployed=', '') as deployed,
replace(regexp_substr(properties, 'type=[^;]*'), 'type=', '') as type,
replace(regexp_substr(properties, 'level=[^;]*'), 'level=', '') as "LEVEL",
replace(regexp_substr(properties, 'description=[^;]*'), 'description=', '') as description,
replace(regexp_substr(properties, 'indicator=[^;]*'), 'indicator=', '') as indicator,
replace(regexp_substr(properties, 'Agreement=[^;]*'), 'Agreement=', '') as Agreement,
replace(regexp_substr(properties, 'Activation date to charge=[^;]*'), 'Activation date to charge=', '') as Activationdatetocharge,
replace(regexp_substr(properties, 'id=[^;]*'), 'id=', '') as id,
replace(regexp_substr(properties, 'name=[^;]*'), 'name=', '') as name,
replace(regexp_substr(properties, 'currencyCode=[^;]*'), 'currencyCode=', '') as currencyCode,
replace(regexp_substr(properties, 'saleExpirationDate=[^;]*'), 'saleExpirationDate=', '') as saleExpirationDate,
replace(regexp_substr(properties, 'Product type=[^;]*'), 'Product type=', '') as Producttype,
replace(regexp_substr(properties, 'saleEffectiveDate=[^;]*'), 'saleEffectiveDate=', '') as saleEffectiveDate,
replace(regexp_substr(properties, 'Deactivation date to charge=[^;]*'), 'Deactivation date to charge=', '') as Deactivationdatetocharge
.
.
.
.
.
.
from OFFER
where name = 'PLAN 599'
;
You have to double-split and generate a query:
Split Row into "ColName=Value"-fields
Split fields into two seperate vars
Create a from-dual-query
DECLARE
inputstring VARCHAR2 (2000) := 'EntityID=1;deployed=2018-01-01;type=app';
myquery VARCHAR2 (2000) := 'SELECT'; -- result-var
tmpValue VARCHAR2 (2000);
tmpName VARCHAR2 (2000);
BEGIN
FOR i IN
(
SELECT TRIM (REGEXP_SUBSTR (inputstring, -- Split input and loop through
'[^;]+',
1,
LEVEL))
l
FROM DUAL
CONNECT BY LEVEL <= REGEXP_COUNT (inputstring, ';') + 1
)
LOOP
tmpName := REGEXP_SUBSTR (i.l, '[^=]+', 1, 1); -- Split column into value and name
tmpValue := REGEXP_SUBSTR (i.l, '[^=]+', 1, 2);
myquery := myquery || ' ''' || tmpValue || ''' as ' || tmpName || ','; -- build some query
END LOOP;
myQuery := SUBSTR (myQuery, 0, LENGTH (myQuery) - 1) || ' FROM DUAL'; -- complete query
DBMS_OUTPUT.put_line (myQuery); --output result
-- Result: SELECT '1' as EntityID, '2018-01-01' as deployed, 'app' as type FROM DUAL
END;
Anyway you'll get a problem with this query if you want to read out the values by code.
Perhaps you could tell us, what you want to do with the data.
I hope, you only want to convert some data from a file. If so you can add another split-loop to split rows.
Example-Function
CREATE OR REPLACE FUNCTION MakeSQL (inputstring VARCHAR2)
RETURN VARCHAR2
IS
myquery VARCHAR2 (2000); -- result-var
tmpValue VARCHAR2 (2000);
tmpName VARCHAR2 (2000);
BEGIN
FOR x IN ( SELECT TRIM (REGEXP_SUBSTR (inputstring, -- Split input and loop through
'[^' || CHR (10) || ']+',
1,
LEVEL))
tmpRow
FROM DUAL
CONNECT BY LEVEL <= REGEXP_COUNT (inputstring, CHR (10)) + 1)
LOOP
myquery := myquery || 'SELECT';
FOR i IN ( SELECT TRIM (REGEXP_SUBSTR (x.tmpRow, -- Split input and loop through
'[^;]+',
1,
LEVEL))
l
FROM DUAL
CONNECT BY LEVEL <= REGEXP_COUNT (x.tmpRow, ';') + 1)
LOOP
tmpName :=
REGEXP_SUBSTR (i.l,
'[^=]+',
1,
1); -- Split column into value and name
tmpValue :=
REGEXP_SUBSTR (i.l,
'[^=]+',
1,
2);
myquery :=
myquery || ' ''' || tmpValue || ''' as ' || tmpName || ','; -- build some query
END LOOP;
myQuery :=
SUBSTR (myQuery, 0, LENGTH (myQuery) - 1)
|| ' FROM DUAL UNION ALL'
|| CHR (10); -- complete row-select
END LOOP;
myQuery := SUBSTR (myQuery, 0, LENGTH (myQuery) - 11); -- complete query
DBMS_OUTPUT.put_line (myQuery); --output result
RETURN myQuery;
END MakeSQL;
Example call
SELECT MakeSQL('EntityID=1;deployed=2018-01-01;type=app
EntityID=2;deployed=2018-02-02;type=app') FROM DUAL;
Example-Result
SELECT '1' as EntityID, '2018-01-01' as deployed, 'app' as type FROM DUAL UNION ALL
SELECT '2' as EntityID, '2018-02-02' as deployed, 'app' as type FROM DUAL
I'm looking for a way to achieve this in a SELECT statement.
FROM
Column1 Column2 Column3
A,B,C 1,2,3 x,y,z
TO
Result
A|1|x,B|2|y,C|3|z
The delimiters don't matter. I'm just trying to to get all the data in one single column. Ideally I am looking to do this in DB2. But I'd like to know if there's an easier way to get this done in Oracle.
Thanks
You can do it like this using INSTR and SUBSTR:
select
substr(column1,1,instr(column1,',',1)-1) || '|' ||
substr(column2,1,instr(column2,',',1)-1) || '|' ||
substr(column3,1,instr(column3,',',1)-1) || '|' ||
',' ||
substr(column1 ,instr(column1 ,',',1,1)+1,instr(column1 ,',',1,2) - instr(column1 ,',',1)-1) || '|' ||
substr(column2 ,instr(column2 ,',',1,1)+1,instr(column2 ,',',1,2) - instr(column2 ,',',1)-1) || '|' ||
substr(column3 ,instr(column3 ,',',1,1)+1,instr(column3 ,',',1,2) - instr(column3 ,',',1)-1) || '|' ||
',' ||
substr(column1 ,instr(column1 ,',',1,2)+1) || '|' ||
substr(column2 ,instr(column2 ,',',1,2)+1) || '|' ||
substr(column3 ,instr(column3 ,',',1,2)+1)
from yourtable
i tried some thing. just look into link
first i created a table called t_ask_test and inserted the data based on the above question. Achieved the result by using the string functions
sample table
create table t_ask_test(column1 varchar(10), column2 varchar(10),column3 varchar(10));
inserted a row
insert into T_ASK_TEST values ('A,B,C','1,2,3','x,y,z');
the following query will be in dynamic way
select substr(column1,1,instr(column1,',',1,1)-1)||'|'||substr(column2,1,instr(column1,',',1,1)-1)||'|'||substr(column3,1,instr(column1,',',1,1)-1) ||','||
substr(column1,instr(column1,',',1,1)+1,instr(column1,',',1,2)-instr(column1,',',1,1)-1)||'|'||substr(column2,instr(column2,',',1,1)+1,instr(column2,',',1,2)-instr(column2,',',1,1)-1)||'|'||substr(column3,instr(column3,',',1,1)+1,instr(column3,',',1,2)-instr(column3,',',1,1)-1) ||','||
substr(column1,instr(column1,',',1,2)+1,length(column1)-instr(column1,',',1,2))||'|'||substr(column2,instr(column2,',',1,2)+1,length(column2)-instr(column2,',',1,2))||'|'||substr(column3,instr(column3,',',1,2)+1,length(column3)-instr(column3,',',1,2)) as test from t_ask_test;
output will be as follows
TEST
---------------
A|1|x,B|2|y,C|3|z
If you have a dynamic number of entries for each row then:
SQL Fiddle
Oracle 11g R2 Schema Setup:
CREATE TABLE TEST ( Column1, Column2, Column3 ) AS
SELECT 'A,B,C', '1,2,3', 'x,y,z' FROM DUAL
UNION ALL SELECT 'D,E', '4,5', 'v,w' FROM DUAL;
Query 1:
WITH ids AS (
SELECT t.*, ROWNUM AS id
FROM TEST t
)
SELECT LISTAGG(
REGEXP_SUBSTR( i.Column1, '[^,]+', 1, n.COLUMN_VALUE )
|| '|' || REGEXP_SUBSTR( i.Column2, '[^,]+', 1, n.COLUMN_VALUE )
|| '|' || REGEXP_SUBSTR( i.Column3, '[^,]+', 1, n.COLUMN_VALUE )
, ','
) WITHIN GROUP ( ORDER BY n.COLUMN_VALUE ) AS value
FROM ids i,
TABLE(
CAST(
MULTISET(
SELECT LEVEL
FROM DUAL
CONNECT BY LEVEL <= GREATEST(
REGEXP_COUNT( i.COLUMN1, '[^,]+' ),
REGEXP_COUNT( i.COLUMN2, '[^,]+' ),
REGEXP_COUNT( i.COLUMN3, '[^,]+' )
)
)
AS SYS.ODCINUMBERLIST
)
) n
GROUP BY i.ID
Results:
| VALUE |
|-------------------|
| A|1|x,B|2|y,C|3|z |
| D|4|v,E|5|w |
You need to use:
SUBSTR
INSTR
|| concatenation operator
It would be easy if you break your output, and then understand how it works.
SQL> WITH t AS
2 ( SELECT 'A,B,C' Column1, '1,2,3' Column2, 'x,y,z' Column3 FROM dual
3 )
4 SELECT SUBSTR(column1, 1, instr(column1, ',', 1) -1)
5 ||'|'
6 || SUBSTR(column2, 1, instr(column2, ',', 1) -1)
7 ||'|'
8 || SUBSTR(column3, 1, instr(column1, ',', 1) -1)
9 ||','
10 || SUBSTR(column1, instr(column1, ',', 1, 2) +1 - instr(column1, ',', 1),
11 instr(column1, ',', 1) -1)
12 ||'|'
13 || SUBSTR(column2, instr(column2, ',', 1, 2) +1 - instr(column2, ',', 1),
14 instr(column2, ',', 1) -1)
15 ||'|'
16 || SUBSTR(column3, instr(column3, ',', 1, 2) +1 - instr(column3, ',', 1),
17 instr(column3, ',', 1) -1)
18 ||','
19 || SUBSTR(column1, instr(column1, ',', 1, 3) +1 - instr(column1, ',', 1),
20 instr(column1, ',', 2) -1)
21 as "new_column"
22 FROM t;
new_column
-------------
A|1|x,B|2|y,C
On a side note, you should avoid storing delimited values in a single column. Consider normalizing the data.
From Oracle 11g and above, you could create a VIRTUAL COLUMN using the above expression and use it instead of executing the SQL frequently.
Its very simple in oracle. just use the concatenation operatort ||.
In the below solution, I have used underscore as the delimiter
select Column1 ||'_'||Column2||'_'||Column3 from table_name;