Hil All,
I have a table , count is about 200M. It has a column which contains data separated by '~'. I want to parse it.
e.g:
Column1
A~B~C~D~E~F
Result :
Column_new1
A~C~E
I just want to skip 2,4,6,n th. words. I don't want plsql. I need sql query. And table is very big,I also need performance.
I use substr,instr functions and I can parse. But it runs really slowly..
Thanks for help.
If you are after performance then use the INSTR and SUBSTR simple string functions:
SELECT SUBSTR(column1, 1, p1 - 1 ) || '~' ||
SUBSTR(column1, p2 + 1, p3 - p2 - 1) || '~' ||
SUBSTR(column1, p4 + 1, p5 - p4 - 1) AS column1_new
FROM (
SELECT column1,
INSTR(column1, '~', 1, 1) AS p1,
INSTR(column1, '~', 1, 2) AS p2,
INSTR(column1, '~', 1, 3) AS p3,
INSTR(column1, '~', 1, 4) AS p4,
INSTR(column1, '~', 1, 5) AS p5
FROM table_name
);
Which, for the sample data:
CREATE TABLE table_name (column1) AS
SELECT 'A~B~C~D~E~F' FROM DUAL;
Outputs:
COLUMN1_NEW
A~C~E
If you want a shorter query then you can use regular expressions:
SELECT REGEXP_REPLACE(column1, '([^~]+)~[^~]+~([^~]+)~[^~]+~([^~]+).*', '\1~\2~\3' )
AS column1_new
FROM table_name;
However, you will find that performance is likely to be an order of magnitude worse than simple string functions.
Another alternative would be to generate a materialized view.
db<>fiddle here
This is a regular expressions option. Looks nice, isn't PL/SQL, works OK (for 2 rows). I'm afraid that anything will run slow for 200 million rows.
SQL> with test (id, col) as
2 (select 1, 'A~B~C~D~E~F' from dual union all
3 select 2, 'M~N~O~P~Q-R' from dual
4 )
5 select id,
6 regexp_substr(col, '\w+', 1, 1) || '~' ||
7 regexp_substr(col, '\w+', 1, 3) || '~' ||
8 regexp_substr(col, '\w+', 1, 5) result
9 from test;
ID RESULT
---------- -----------------------------------
1 A~C~E
2 M~O~Q
SQL>
I have a Strings like:
5.3.60.8
6.0.5.94
3.3.4.1
How to sort these values in sorting order in Oracle SQL?
I want the order to be like this:
6.0.5.94
5.3.60.8
3.3.4.1
with
inputs ( str ) as (
select '6.0.5.94' from dual union all
select '5.3.60.8' from dual union all
select '3.3.4.1' from dual
)
select str from inputs
order by to_number(regexp_substr(str, '\d+', 1, 1)),
to_number(regexp_substr(str, '\d+', 1, 2)),
to_number(regexp_substr(str, '\d+', 1, 3)),
to_number(regexp_substr(str, '\d+', 1, 4))
;
STR
--------
3.3.4.1
5.3.60.8
6.0.5.94
You could pad numbers with zeroes on the left in the order by clause:
select version
from versions
order by regexp_replace(
regexp_replace(version, '(\d+)', lpad('\1', 11, '0')),
'\d+(\d{10})',
'\1'
) desc
This works for more number parts as well, up to about 200 of them.
If you expect to have numbers with more than 10 digits, increase the number passed as second argument to the lpad function, and also the braced number in the second regular expression. The first should be one more (because \1 is two characters but could represent only one digit).
Highest version
To get the highest version only, you can add the row number to the query above with the special Oracle rownum keyword. Then wrap all that in an another select with a condition on that row number:
select version
from (
select version, rownum as row_num
from versions
order by regexp_replace(
regexp_replace(version, '(\d+)', lpad('\1', 11, '0')),
'\d+(\d{10})',
'\1'
) desc)
where row_num <= 1;
See this Q&A for several alternatives, also depending on your Oracle version.
I will show here the answer from AskTom, which can be used with different version size :
WITH inputs
AS (SELECT 1 as id, '6.0.5.94' as col FROM DUAL
UNION ALL
SELECT 2,'5.3.30.8' FROM DUAL
UNION ALL
SELECT 3,'5.3.4.8' FROM DUAL
UNION ALL
SELECT 4,'3' FROM DUAL
UNION ALL
SELECT 5,'3.3.40' FROM DUAL
UNION ALL
SELECT 6,'3.3.4.1.5' FROM DUAL
UNION ALL
SELECT 7,'3.3.4.1' FROM DUAL)
SELECT col, MAX (SYS_CONNECT_BY_PATH (v, '.')) p
FROM (SELECT t.col, TO_NUMBER (SUBSTR (x.COLUMN_VALUE, 1, 5)) r, SUBSTR (x.COLUMN_VALUE, 6) v, id rid
FROM inputs t,
TABLE (
CAST (
MULTISET (
SELECT TO_CHAR (LEVEL, 'fm00000')
|| TO_CHAR (TO_NUMBER (SUBSTR ('.' || col || '.', INSTR ('.' || col || '.', '.', 1, ROWNUM) + 1, INSTR ('.' || col || '.', '.', 1, ROWNUM + 1) - INSTR ('.' || col || '.', '.', 1, ROWNUM) - 1)), 'fm0000000000')
FROM DUAL
CONNECT BY LEVEL <= LENGTH (col) - LENGTH (REPLACE (col, '.', '')) + 1) AS SYS.odciVarchar2List)) x)
START WITH r = 1
CONNECT BY PRIOR rid = rid AND PRIOR r + 1 = r
GROUP BY col
ORDER BY p
I'm using this SQL query to generate random value in PostgreSQL
chr(ascii('B') + (random() * 25)::integer)
How I can generate 15 characters random String using the same query?
Another solution that's pretty easy to read (perf should be reasonable, but no benchmarks were performed):
select substr(md5(random()::text), 0, 25);
Could be uppercased if you prefer:
select upper(substr(md5(random()::text), 0, 25));
Here is my contrib
postgres=# SELECT array_to_string(array(select substr('ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789',((random()*(36-1)+1)::integer),1) from generate_series(1,50)),'');
array_to_string
----------------------------------------------------
4XOS6TQG5JORLF3D1RPXUWR2FQKON9HIXV0UGH0CQFT1LN5D4L
(1 row)
It lets you specify the set of allowed characters and the length of the string.
This will give you a random word of length 15 consisting of the letters configured in the source values constant
select
string_agg(substr(characters, (random() * length(characters) + 1)::integer, 1), '') as random_word
from (values('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789')) as symbols(characters)
-- length of word
join generate_series(1, 15) on 1 = 1
EDIT: to obtain multiple random words you can use the following:
with symbols(characters) as (VALUES ('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789'))
select string_agg(substr(characters, (random() * length(characters) + 1) :: INTEGER, 1), '')
from symbols
join generate_series(1,8) as word(chr_idx) on 1 = 1 -- word length
join generate_series(1,10000) as words(idx) on 1 = 1 -- # of words
group by idx
Yes can do this by single query also but if you want every char should be separate according to range then above is solution
SELECT array_to_string(ARRAY(
SELECT chr((ascii('B') + round(random() * 25)) :: integer)
FROM generate_series(1,15)),
'');
I tried to use solution from #Bennit
but noticed some flaws. The random part is calculated a bit wrongly, that leads wrong results: the resulting lenght is differs (shorter) than desired.
[took a quick look at the #lyndon-s version - most probably it also has the same drawback]
So here is updated version of #bennit version:
select
string_agg(substr(characters, (random() * length(characters) + 0.5)::integer, 1), '') as random_word
from (values('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789')) as symbols(characters)
-- length of word
join generate_series(1, 15) on 1 = 1
And here is demonstration why the change is required:
Corrected:
select n,count(*) from (
select (random() * 10 + 0.5)::integer as n from dbfiles
join generate_series(1, 100000) on 1 = 1
) as s group by n
order by 1;
Original:
select n,count(*) from (
select (random() * 10 + 1)::integer as n from dbfiles
join generate_series(1, 100000) on 1 = 1
) as s group by n
order by 1;
I use this, for generating random strings...
If you dont mind dashes and have the uuid extension enabled...
select substr(uuid_generate_v4()::text,1,15);
e.g. to generate a random string in the name column, i will use
select concat('name-', substr(uuid_generate_v4()::text,1,10)) as name;
e.g. name-91fc72dc-d
else, use the excellent md5 example from #fncomp
nb: To enable the uuid extension
create extension if not exists "uuid-ossp";
Here is the idea:
select (chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer) ||
chr(ascii('B') + (random() * 25)::integer)
) as Random15
For me the most convenient way is to create a function:
CREATE OR REPLACE FUNCTION random_string(int) RETURNS TEXT as $$
SELECT substr(md5(random()::text), 0, $1+1);
$$ language sql;
The function is named random_string
and It takes string length as an argument.
And then I use this function anywhere I want
Just see the result:
select random_string(6);
Make a single insert:
insert into users values(nextval('users_sequence'), random_string(6), random_string(6));
Generate multiple rows with random data:
do $$
begin
for i in 1..100 loop
insert into users values(nextval('users_sequence'), random_string(6), random_string(6));
end loop;
end;
$$;
and so on.
Throwing in my 2c here. I needed random strings to do some benchmarks, so all that really mattered to me was that the strings were unique from each other.
select rpad(generate_series::varchar, 1000, 'hi') from generate_series(1,10);
rpad - pad right till length(1000), padded with 'hi'
generate_series(1,10) - generate 10 rows
Combining with an answer above, you could also do this:
select rpad(generate_series::varchar, 1000, md5(random()::text)) from generate_series(1,10)
That makes sure you have 200 chars, or whatever the desired length is.
Well how about a recursive CTE. Combine with generate series to get however many you want.
with recursive brs(rstg, n) as
( select chr(ascii('B') + (random() * 25)::integer), 1
from generate_series(1,50) --- or however many you want
union all
select rstg || chr(ascii('B') + (random() * 25)::integer), n+1
from brs
where n<= 15
)
select rstg
from brs
where n=15;
Here's my PL/pgSQL take:
there is an option to generate texts in english or russian symbols, and it's easy to expand for more languages;
optional usage of digits, punctuation symbols, whitespaces and linebreaks;
create or replace function random_string (
str_length integer, lang varchar(2) default 'en',
w_dig boolean default true, w_punct boolean default true,
w_space boolean default false, w_newline boolean default false
)
returns text
language plpgsql
as $function$
declare
chars_eng text[] := '{A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y,Z,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}';
chars_rus text[] := '{А,Б,В,Г,Д,Е,Ё,Ж,З,И,Й,К,Л,М,Н,О,П,Р,С,Т,У,Ф,Х,Ц,Ч,Ш,Щ,Ъ,Ы,Ь,Э,Ю,Я,а,б,в,г,д,е,ё,ж,з,и,й,к,л,м,н,о,п,р,с,т,у,ф,х,ц,ч,ш,щ,ъ,ы,ь,э,ю,я}';
chars_dig text[] := '{}';
chars_punct text[] := '{}';
chars_space text[] := '{}';
chars_newline text[] := '{}';
chars_final text[] := '{}';
result text := '';
i integer := 0;
begin
-- checking string length arg
if str_length < 0 then
raise exception 'Length of string cannot be a negative value';
end if;
-- checking chars selection
if w_dig = true then
chars_dig := '{0,1,2,3,4,5,6,7,8,9}';
end if;
if w_punct = true then
chars_punct := string_to_array(E'!d"d#d$d%d&d\'d(d)d*d+d,d-d.d/d:d;d<d=d>d?d#d[d\\d]d^d_d`d{d|d}d~','d');
end if;
if w_space = true then
chars_space := string_to_array(' ',',');
end if;
if w_newline = true then
chars_newline := string_to_array(E'\r\n',',');
end if;
-- checking language selection
if lang = 'en' then
chars_final := chars_eng||chars_dig||chars_punct||chars_space||chars_newline;
elsif lang = 'ru' then
chars_final := chars_rus||chars_dig||chars_punct||chars_space||chars_newline;
else
raise exception 'Characters set for that language is not defined';
end if;
-- filling the string
for i in 1..str_length loop
result := result || chars_final[1 + round(random() * (array_length(chars_final, 1) - 1))];
end loop;
-- trimming extra symbols that may appear from /r/n usage
if length(result) > str_length then
result := left(result, str_length);
end if;
-- getting the result
return result;
end;
$function$ ;