Order by generated column - sql

I have following data structure:
CREATE TABLE test(
id INTEGER PRIMARY KEY,
data TEXT NOT NULL
);
INSERT INTO test (data) VALUES ('10.20.3.40'), ('10.100.3'), ('10.20.20.40')
The problem is that i need to order by data column using integer logic (dot-separated string as array of integers).
Using order by it returns data sorted as text:
SELECT data FROM test ORDER BY data
10.100.3
10.20.20.40
10.20.3.40
Result I need to achieve:
10.20.3.40
10.20.20.40
10.100.3
The simplest method to sort it properly without reimplementing arrays in SQLite which I've found is to add zero-padding to each part of data.
So basically I need to:
Select all data from table;
Split value of data column;
Add zero-padding and join it back;
Join newly generated column with reformatted data
Order by this column
What have I already done:
WITH RECURSIVE split_str(source, part) AS (
SELECT '10.20.3.40' || '.', NULL
UNION ALL
SELECT
substr(source, instr(source, '.') + 1),
substr('000000' || source, instr(source, '.'), 6)
FROM split_str WHERE source != ''
)
SELECT group_concat(part, '.') AS new_data FROM split_str WHERE part IS NOT NULL
It splits constant string '10.20.3.40' by dot, add leading zeros to each part and join it back using group_concat(). It returns:
000010.000020.000003.000040
Now I need to apply such a modification to values of data column from test table and somehow use this values for sorting. That's result I'm trying to get:
I'm not an expert in SQL (obviously) and don't understand how to apply expression in WITH clause on each data column separately.

As you can't use GROUP_CONCAT() if you want to preserve the order, just build up another string instead.
Then, only take the records where there's no more 'unpadded' string still to be processed.
WITH RECURSIVE
test_set(original)
AS
(
SELECT '10.20.3.40'
UNION ALL
SELECT '10.100.3'
UNION ALL
SELECT '10.20.20.40'
),
split_str(original, remaining, padded)
AS
(
SELECT original, original || '.', '' FROM test_set
---------
UNION ALL
---------
SELECT
original,
substr(remaining, instr(remaining, '.') + 1),
padded || '.' || substr('000000' || remaining, instr(remaining, '.'), 6)
FROM
split_str
WHERE
remaining != ''
)
SELECT
original,
padded
FROM
split_str
WHERE
remaining = ''
ORDER BY
padded
Demo: db<>fiddle.uk
(You may or may not want to strip the leading ., depending on your needs.)

Related

Remove items in a delimited list that are non numeric in SQL for Redshift

I am working with a field called codes that is a delimited list of values, separated by commas. Within each item there is a title ending in a colon and then a code number following the colon. I want a list of only the code numbers after each colon.
Example Value:
name-form-na-stage0:3278648990379886572,rules-na-unwanted-sdfle2:6886328308933282817,us-disdg-order-stage1:1273671130817907765
Desired Output:
3278648990379886572,6886328308933282817,1273671130817907765
The title does always start with a letter and the end with a colon so I can see how REGEXP_REPLACE might work to replace any string between starting with a letter and ending with a colon with '' might work but I am not good at REGEXP_REPLACE patterns. Chat GPT is down fml.
Side note, if anyone knows of a good guide for understanding pattern notation for regular expressions it would be much appreciated!
I tried this and it is not working REGEXP_REPLACE(REPLACE(REPLACE(codes,':', ' '), ',', ' ') ,' [^0-9]+ ', ' ')
This solution assumes a few things:
No colons anywhere else except immediately before the numbers
No number at the very start
At a high level, this query finds how many colons there are, splits the entire string into that many parts, and then only keeps the number up to the comma immediately after the number, and then aggregates the numbers into a comma-delimited list.
Assuming a table like this:
create temp table tbl_string (id int, strval varchar(1000));
insert into tbl_string
values
(1, 'name-form-na-stage0:3278648990379886572,rules-na-unwanted-sdfle2:6886328308933282817,us-disdg-order-stage1:1273671130817907765');
with recursive cte_num_of_delims AS (
select max(regexp_count(strval, ':')) AS num_of_delims
from tbl_string
), cte_nums(nums) AS (
select 1 as nums
union all
select nums + 1
from cte_nums
where nums <= (select num_of_delims from cte_num_of_delims)
), cte_strings_nums_combined as (
select id,
strval,
nums as index
from cte_nums
cross join tbl_string
), prefinal as (
select *,
split_part(strval, ':', index) as parsed_vals
from cte_strings_nums_combined
where parsed_vals != ''
and index != 1
), final as (
select *,
case
when charindex(',', parsed_vals) = 0
then parsed_vals
else left(parsed_vals, charindex(',', parsed_vals) - 1)
end as final_vals
from prefinal
)
select listagg(final_vals, ',')
from final

Search a pattern from comma seperated parameters in plsql

My Parameter to a procedure lv_ip := 'MNS-GC%|CS,MIB-TE%|DC'
My cursor query should search for records that start with 'MNS-GC%' and 'MIB-TE%'.
Select id, date,program,program_start_date
from table_1
where program like 'MNS-GC%' or program LIKE 'MIB-TE%'
Please suggest ways to read it from the parameter and an alternative to LIKE.
Since you mention you want to preserve what's on the right side of the pipe, and want to be able to process parameters dynamically, here's a way to parse multi-delimited data that could give you some ideas using a CTE.
The table called 'tbl' just sets up your original data. tbl_comma contains that data split on the comma. The final query splits that data into name/value pairs.
Hopefully this will help give you some ideas even though it's not the exact answer you are looking for.
COLUMN ID FORMAT a3
COLUMN PROGRAM FORMAT a10
COLUMN part2 FORMAT a6
-- Original data
WITH tbl(ID, DATA) AS (
SELECT 1, 'MNS-GC%|CS,MIB-TE%|DC' FROM dual UNION ALL
SELECT 2, 'MNS-GC%|CS,MIB-TE%|DC,MIB-TA%|AB,MIB-TB%|BC' FROM dual
),
tbl_comma(ID, CASE) AS (
SELECT ID,
REGEXP_SUBSTR(DATA, '(.*?)(,|$)', 1, LEVEL, NULL, 1) CASE
FROM tbl
CONNECT BY REGEXP_SUBSTR(DATA, '(.*?)(,|$)', 1, LEVEL) IS NOT NULL
AND PRIOR ID = ID
AND PRIOR SYS_GUID() IS NOT NULL
)
--SELECT * FROM tbl_comma;
-- Parse into name/value pairs
SELECT ID,
REGEXP_REPLACE(CASE, '^(.*)\|.*', '\1') PROGRAM,
REGEXP_REPLACE(CASE, '.*\|(.*)$', '\1') PART2
FROM tbl_comma;
ID PROGRAM PART2
--- ---------- ------
1 MNS-GC% CS
1 MIB-TE% DC
2 MNS-GC% CS
2 MIB-TE% DC
2 MIB-TA% AB
2 MIB-TB% BC
6 rows selected.
If you're stuck with that input and the structure is fixed, with each comma-separated element having a pipe-delimited value, you could possibly convert that string to a regular expression pattern, and then use regexp_like to pattern-match:
select id, date, program, program_start_date
from table_1
where regexp_like(
program,
'^(' || rtrim(regexp_replace(lv_ip, '%\|.*?(,|$)', '|'), '|') || ')')
With your example parameter, the
'^(' || rtrim(regexp_replace(lv_ip, '%\|.*?(,|$)', '|'), '|') || ')'
would generate the pattern
^(MNS-GC|MIB-TE)
i.e. looking for either of those strings at the start of the program value.
db<>fiddle
Alternatively you could split the input up yourself, with instr and substr, and - since the number of elements may vary - create a dynamic query using them. That might be faster than using regular expression, but might be harder to maintain.
What would the regexp be to match CS|DC
It depends how you plan to use those values, but if you're looking for some column exactly matching one of them, then you could do something similar with:
'^(' || ltrim(regexp_replace(l_ip, '(^|,)[^|]*', null), '|') || ')$'
which with your input string would generate the pattern
^(CS|DC)$
But if you need to match the corresponding values as pairs - so the equivalent of something like:
where (program like 'MNS-GC%' and some_col = 'CS')
or (program like 'MIB-TE%' and some_col = 'DC')
... then you'd need to extract them as pairs, as #Gary_W has shown.

How to get maximum value of a specific part of strings?

I have below records
Id Title
500006 FS/97/98/037
500007 FS/97/04/035
500008 FS/97/01/036
500009 FS/97/104/040
I should split Title field and get 4th part of text and return maximum value. In this example my query should return 040 or 40.
select max(cast(right(Title, charindex('/', reverse(Title) + '/') - 1) as int))
from your_table
SQLFiddle demo
You can use PARSENAME function since you always have 4 parts(confirmed in comments section)
select max(cast(parsename(replace(Title,'/','.'),1) as int))
from yourtable
If you want to split the data in the Title column and get the part from the splitted text by position, you may try with one JSON-based approach with a simple string transformation. You need to transform the data in the Title column into a valid JSON array (FS/97/98/037 into ["FS","97","08","037"]) and after that to parse thе data with OPENJSON(). The result from OPENJSON() (using default schema and parsing JSON array) is a table with columns key, value and type, and the key column holds the index of the items in the JSON array:
Note, that using STRING_SPLIT() is not an option here, because the order of the returned rows is not guaranteed.
Table:
CREATE TABLE Data (
Id varchar(6),
Title varchar(50)
)
INSERT INTO Data
(Id, Title)
VALUES
('500006', 'FS/97/98/037'),
('500007', 'FS/97/04/035'),
('500008', 'FS/97/01/036'),
('500009', 'FS/97/104/040')
Statement:
SELECT MAX(j.[value])
FROM Data d
CROSS APPLY OPENJSON(CONCAT('["', REPLACE(d.Title, '/', '","'), '"]')) j
WHERE (j.[key] + 1) = 4
If you data has fixed format with 4 parts, even this approach may help:
SELECT MAX(PARSENAME(REPLACE(Title, '/', '.'), 1))
FROM Data
You can also try the below query.
SELECT Top 1
CAST('<x>' + REPLACE(Title,'/','</x><x>') + '</x>' AS XML).value('/x[4]','int') as Value
from Data
order by 1 desc
You can find the live demo Here.

How to remove specific value from comma separated string in oracle

I want remove specific value from comma separated sting using oracle.
Sample Input -
col
1,2,3,4,5
Suppose i want to remove 3 from the string.
Sample Output -
col
1,2,4,5
Please suggest how i can do this using oracle query.
Thanks.
Here is a solution that uses only standard string functions (rather than regular expressions) - which should result in faster execution in most cases; it removes 3 only when it is the first character followed by comma, the last character preceded by comma, or preceded and followed by comma, and it removes the comma that precedes it in the middle case and it removes the comma that follows it in the first and third case.
It is able to remove two 3's in a row (which some of the other solutions offered are not able to do) while leaving in place consecutive commas (which presumably stand in for NULL) and do not disturb numbers like 38 or 123.
The strategy is to first double up every comma (replace , with ,,) and append and prepend a comma (to the beginning and the end of the string). Then remove every occurrence of ,3,. From what is left, replace every ,, back with a single , and finally remove the leading and trailing ,.
with
test_data ( str ) as (
select '1,2,3,4,5' from dual union all
select '1,2,3,3,4,4,5' from dual union all
select '12,34,5' from dual union all
select '1,,,3,3,3,4' from dual
)
select str,
trim(both ',' from
replace( replace(',' || replace(str, ',', ',,') || ',', ',3,'), ',,', ',')
) as new_str
from test_data
;
STR NEW_STR
------------- ----------
1,2,3,4,5 1,2,4,5
1,2,3,3,4,4,5 1,2,4,4,5
12,34,5 12,34,5
1,,,3,3,3,4 1,,,4
4 rows selected.
Note As pointed out by MT0 (see Comments below), this will trim too much if the original string begins or ends with commas. To cover that case, instead of wrapping everything within trim(both ',' from ...) I should wrap the rest within a subquery, and use something like substr(new_str, 2, length(new_str) - 2) in the outer query.
Here is one method:
select trim(both ',' from replace(',' || '1,2,3,4,5' || ',', ',' || '3' || ',', ','))
That said, storing comma-delimited strings is a really, really bad idea. There is almost no reason to do such a thing. Oracle supports JSON, XML, and nested tables -- all of which are better alternatives.
The need to remove an element suggests a poor data design.
You can convert the list rows using an XMLTABLE, filter to remove the unwanted rows and then re-aggregate them:
SELECT LISTAGG( x.value.getStringVal(), ',' ) WITHIN GROUP ( ORDER BY idx )
FROM XMLTABLE(
( '1,2,3,4,5' )
COLUMNS value XMLTYPE PATH '.',
idx FOR ORDINALITY
) x
WHERE x.value.getStringVal() != 3;
For a simple filter this is probably not worth it and you should use something like (based on #mathguy's solution):
SELECT SUBSTR( new_list, 2, LENGTH( new_list ) - 2 ) AS new_list
FROM (
SELECT REPLACE(
REPLACE(
',' || REPLACE( :list, ',', ',,' ) || ',',
',' || :value_to_replace || ','
),
',,',
','
) AS new_list
FROM DUAL
)
However, if the filtering is more complicated then it might be worth converting the list to rows, filtering and re-aggregating.
I do not knwo how to do this in Oracle, but with SQL-Server I'd use a trick:
convert the list to XML by replacing the comma with tags
use XQuery to filter the data
reconcatenate
This is SQL Server syntax but might point you the direction:
declare #s varchar(100)='1,2,2,3,3,4';
declare #exclude int=3;
WITH Casted AS
(
SELECT CAST('<x>' + REPLACE(#s,',','</x><x>') + '</x>' AS XML) AS TheXml
)
SELECT x.value('.','int')
FROM Casted
CROSS APPLY TheXml.nodes('/x[text()!=sql:variable("#exclude")]') AS A(x)
UPDATE
I just found this answer which seems to show pretty well how to start...
I agree with Gordon regarding the fact that storing comma delimited data in a column is a really bad idea.
I just preceed the csv with a ',', then use the replace function followed by a left trim function to clean-up the preceeding ','.
SCOTT#tst>VAR b_number varchar2(5);
SCOTT#tst>EXEC :b_number:= '3';
PL/SQL procedure successfully completed.
SCOTT#tst>WITH srce AS (
2 SELECT
3 ',' || '3,1,2,3,3,4,5,3' col
4 FROM
5 dual
6 ) SELECT
7 ltrim(replace(col,',' ||:b_number),',') col
8 FROM
9 srce;
COL
1,2,4,5

Teradata : Sum up values in a column

Problem Statement
Example is shown in below image :
The last 2 rows have the patterns like "1.283 2 3" in a single cell. The numbers are seperated by space in the column. We need to add those nos and represent in the format given in Output.
So, the cell having "1.283 2 3" must be converted to 6.283
Challenges facing :
The column values are in string format.
Add nos after casting them into integer
Donot want to take data in UNIX box and manipulate the same.
In TD14 there would be a built-in table UDF named STRTOK_SPLIT_TO_TABLE, before you need to implement your own UDF or use a recursive query.
I modified an existing string splitting script to use blanks as delimiter:
CREATE VOLATILE TABLE Strings
(
groupcol INT NOT NULL,
string VARCHAR(991) NOT NULL
) ON COMMIT PRESERVE ROWS;
INSERT INTO Strings VALUES (1,'71.792');
INSERT INTO Strings VALUES (2,'71.792 1 2');
INSERT INTO Strings VALUES (3,'1.283 2 3');
WITH RECURSIVE cte
(groupcol,
--string,
len,
remaining,
word,
pos
) AS (
SELECT
GroupCol,
--String,
POSITION(' ' IN String || ' ') - 1 AS len,
TRIM(LEADING FROM SUBSTRING(String || ' ' FROM len + 2)) AS remaining,
TRIM(SUBSTRING(String FROM 1 FOR len)) AS word,
1
FROM strings
UNION ALL
SELECT
GroupCol,
--String,
POSITION(' ' IN remaining)- 1 AS len_new,
TRIM(LEADING FROM SUBSTRING(remaining FROM len_new + 2)),
TRIM(SUBSTRING(remaining FROM 1 FOR len_new)),
pos + 1
FROM cte
WHERE remaining <> ''
)
SELECT
groupcol,
-- remove the NULLIF to get 0 for blank strings
SUM(CAST(NULLIF(word, '') AS DECIMAL(18,3)))
FROM cte
GROUP BY 1
This might use a lot of spool, hopefully you're not running that on a large table.