Teradata : Sum up values in a column - sql

Problem Statement
Example is shown in below image :
The last 2 rows have the patterns like "1.283 2 3" in a single cell. The numbers are seperated by space in the column. We need to add those nos and represent in the format given in Output.
So, the cell having "1.283 2 3" must be converted to 6.283
Challenges facing :
The column values are in string format.
Add nos after casting them into integer
Donot want to take data in UNIX box and manipulate the same.

In TD14 there would be a built-in table UDF named STRTOK_SPLIT_TO_TABLE, before you need to implement your own UDF or use a recursive query.
I modified an existing string splitting script to use blanks as delimiter:
CREATE VOLATILE TABLE Strings
(
groupcol INT NOT NULL,
string VARCHAR(991) NOT NULL
) ON COMMIT PRESERVE ROWS;
INSERT INTO Strings VALUES (1,'71.792');
INSERT INTO Strings VALUES (2,'71.792 1 2');
INSERT INTO Strings VALUES (3,'1.283 2 3');
WITH RECURSIVE cte
(groupcol,
--string,
len,
remaining,
word,
pos
) AS (
SELECT
GroupCol,
--String,
POSITION(' ' IN String || ' ') - 1 AS len,
TRIM(LEADING FROM SUBSTRING(String || ' ' FROM len + 2)) AS remaining,
TRIM(SUBSTRING(String FROM 1 FOR len)) AS word,
1
FROM strings
UNION ALL
SELECT
GroupCol,
--String,
POSITION(' ' IN remaining)- 1 AS len_new,
TRIM(LEADING FROM SUBSTRING(remaining FROM len_new + 2)),
TRIM(SUBSTRING(remaining FROM 1 FOR len_new)),
pos + 1
FROM cte
WHERE remaining <> ''
)
SELECT
groupcol,
-- remove the NULLIF to get 0 for blank strings
SUM(CAST(NULLIF(word, '') AS DECIMAL(18,3)))
FROM cte
GROUP BY 1
This might use a lot of spool, hopefully you're not running that on a large table.

Related

Remove items in a delimited list that are non numeric in SQL for Redshift

I am working with a field called codes that is a delimited list of values, separated by commas. Within each item there is a title ending in a colon and then a code number following the colon. I want a list of only the code numbers after each colon.
Example Value:
name-form-na-stage0:3278648990379886572,rules-na-unwanted-sdfle2:6886328308933282817,us-disdg-order-stage1:1273671130817907765
Desired Output:
3278648990379886572,6886328308933282817,1273671130817907765
The title does always start with a letter and the end with a colon so I can see how REGEXP_REPLACE might work to replace any string between starting with a letter and ending with a colon with '' might work but I am not good at REGEXP_REPLACE patterns. Chat GPT is down fml.
Side note, if anyone knows of a good guide for understanding pattern notation for regular expressions it would be much appreciated!
I tried this and it is not working REGEXP_REPLACE(REPLACE(REPLACE(codes,':', ' '), ',', ' ') ,' [^0-9]+ ', ' ')
This solution assumes a few things:
No colons anywhere else except immediately before the numbers
No number at the very start
At a high level, this query finds how many colons there are, splits the entire string into that many parts, and then only keeps the number up to the comma immediately after the number, and then aggregates the numbers into a comma-delimited list.
Assuming a table like this:
create temp table tbl_string (id int, strval varchar(1000));
insert into tbl_string
values
(1, 'name-form-na-stage0:3278648990379886572,rules-na-unwanted-sdfle2:6886328308933282817,us-disdg-order-stage1:1273671130817907765');
with recursive cte_num_of_delims AS (
select max(regexp_count(strval, ':')) AS num_of_delims
from tbl_string
), cte_nums(nums) AS (
select 1 as nums
union all
select nums + 1
from cte_nums
where nums <= (select num_of_delims from cte_num_of_delims)
), cte_strings_nums_combined as (
select id,
strval,
nums as index
from cte_nums
cross join tbl_string
), prefinal as (
select *,
split_part(strval, ':', index) as parsed_vals
from cte_strings_nums_combined
where parsed_vals != ''
and index != 1
), final as (
select *,
case
when charindex(',', parsed_vals) = 0
then parsed_vals
else left(parsed_vals, charindex(',', parsed_vals) - 1)
end as final_vals
from prefinal
)
select listagg(final_vals, ',')
from final

In SQL Server, how can I search for column with 1 or 2 whitespace characters?

So I need to filter column which contains either one, two or three whitespace character.
CREATE TABLE a
(
[col] [char](3) NULL,
)
and some inserts like
INSERT INTO a VALUES (' ',' ', ' ')
How do I get only the row with one white space?
Simply writing
SELECT *
FROM a
WHERE column = ' '
returns all rows irrespective of one or more whitespace character.
Is there a way to escape the space? Or search for specific number of whitespaces in column? Regex?
Use like clause - eg where column like '%[ ]%'
the brackets are important, like clauses provide a very limited version of regex. If its not enough, you can add a regex function written in C# to the DB and use that to check each row, but it won't be indexed and thus will be very slow.
The other alternative, if you need speed, is to look into full text search indexes.
Here is one approach you can take:
DECLARE #data table ( txt varchar(50), val varchar(50) );
INSERT INTO #data VALUES ( 'One Space', ' ' ), ( 'Two Spaces', ' ' ), ( 'Three Spaces', ' ' );
;WITH cte AS (
SELECT
txt,
DATALENGTH ( val ) - ( DATALENGTH ( REPLACE ( val, ' ', '' ) ) ) AS CharCount
FROM #data
)
SELECT * FROM cte WHERE CharCount = 1;
RETURNS
+-----------+-----------+
| txt | CharCount |
+-----------+-----------+
| One Space | 1 |
+-----------+-----------+
You need to use DATALENGTH as LEN ignores trailing blank spaces, but this is a method I have used before.
NOTE:
This example assumes the use of a varchar column.
Trailing spaces are often ignored in string comparisons in SQL Server. They are treated as significant on the LHS of the LIKE though.
To search for values that are exactly one space you can use
select *
from a
where ' ' LIKE col AND col = ' '
/*The second predicate is required in case col contains % or _ and for index seek*/
Note with your example table all the values will be padded out to three characters with trailing spaces anyway though. You would need a variable length datatype (varchar/nvarchar) to avoid this.
The advantage this has over checking value + DATALENGTH is that it is agnostic to how many bytes per character the string is using (dependant on datatype and collation)
DB Fiddle
How to get only rows with one space?
SELECT *
FROM a
WHERE col LIKE SPACE(1) AND col NOT LIKE SPACE(2)
;
Though this will only work for variable length datatypes.
Thanks guys for answering.
So I converted the char(3) column to varchar(3).
This seemed to work for me. It seems sql server has ansi padding that puts three while space in char(3) column for any empty or single space input. So any search or len or replace will take the padded value.

Split strings in a column based on text values and numerical values such as patindex

I have a column that displays stock market options data like below:
GME240119C00020000
QQQ240119C00305000
NFLX240119P00455000
I want to be able to split these up so they show up like:
GME|240119|C|00020000
QQQ|240119|C|00305000
NFLX|240119|P|00455000
I was able to split the first portion with the ticker name by using the code below, but I don't know how to split the rest of the strings.
case patindex('%[0-9]%', str)
when 0 then str
else left(str, patindex('%[0-9]%', str) -1 )
end
from t
edit: for anyone who is wondering, I used Dale's solution below to get my desired outcome. I edited the query he provided to make the parts show up as individual columns
select
substring(T.contractSymbol,1,C1.Position-1) as a
,substring(T.contractSymbol,C1.Position,6) as b
,substring(S1.Part,1,1) as c
,substring(S1.Part,2,len(S1.Part)) as d
from Options_Data_All T
cross apply (
values (patindex('%[0-9]%', T.contractSymbol))
) C1 (Position)
cross apply (
values (substring(contractSymbol, C1.Position+6, len(T.contractSymbol)))
) S1 (Part);
Just keep doing what you started doing by using SUBSTRING. So as you did find the first number and actually in your case, based on the data provided, everything else is fixed length, so you don't have to search anymore, just split the string.
declare #Test table (Contents nvarchar(max));
insert into #Test (Contents)
values
('GME240119C00020000'),
('QQQ240119C00305000'),
('NFLX240119P00455000');
select
substring(T.Contents,1,C1.Position-1) + '|' + substring(T.Contents,C1.Position,6) + '|' + substring(S1.Part,1,1) + '|' + substring(S1.Part,2,len(S1.Part))
from #Test T
cross apply (
values (patindex('%[0-9]%', T.Contents))
) C1 (Position)
cross apply (
values (substring(Contents, C1.Position+6, len(T.Contents)))
) S1 (Part);
Returns:
Data
GME|240119|C|00020000
QQQ|240119|C|00305000
NFLX|240119|P|00455000
If one can assume that all but the first column are fixed width then a simple SUBSTRING solution would suffice e.g.
select
substring(Contents,1,len(Contents)-15)
+ '|' + substring(Contents,len(Contents)-14,6)
+ '|' + substring(Contents,len(Contents)-8,1)
+ '|' + substring(Contents,len(Contents)-7,8) [Data]
from #Test;
Note: CROSS APPLY is just a fancy way to use a sub-query to avoid needing to repeat a calculation.

Order by generated column

I have following data structure:
CREATE TABLE test(
id INTEGER PRIMARY KEY,
data TEXT NOT NULL
);
INSERT INTO test (data) VALUES ('10.20.3.40'), ('10.100.3'), ('10.20.20.40')
The problem is that i need to order by data column using integer logic (dot-separated string as array of integers).
Using order by it returns data sorted as text:
SELECT data FROM test ORDER BY data
10.100.3
10.20.20.40
10.20.3.40
Result I need to achieve:
10.20.3.40
10.20.20.40
10.100.3
The simplest method to sort it properly without reimplementing arrays in SQLite which I've found is to add zero-padding to each part of data.
So basically I need to:
Select all data from table;
Split value of data column;
Add zero-padding and join it back;
Join newly generated column with reformatted data
Order by this column
What have I already done:
WITH RECURSIVE split_str(source, part) AS (
SELECT '10.20.3.40' || '.', NULL
UNION ALL
SELECT
substr(source, instr(source, '.') + 1),
substr('000000' || source, instr(source, '.'), 6)
FROM split_str WHERE source != ''
)
SELECT group_concat(part, '.') AS new_data FROM split_str WHERE part IS NOT NULL
It splits constant string '10.20.3.40' by dot, add leading zeros to each part and join it back using group_concat(). It returns:
000010.000020.000003.000040
Now I need to apply such a modification to values of data column from test table and somehow use this values for sorting. That's result I'm trying to get:
I'm not an expert in SQL (obviously) and don't understand how to apply expression in WITH clause on each data column separately.
As you can't use GROUP_CONCAT() if you want to preserve the order, just build up another string instead.
Then, only take the records where there's no more 'unpadded' string still to be processed.
WITH RECURSIVE
test_set(original)
AS
(
SELECT '10.20.3.40'
UNION ALL
SELECT '10.100.3'
UNION ALL
SELECT '10.20.20.40'
),
split_str(original, remaining, padded)
AS
(
SELECT original, original || '.', '' FROM test_set
---------
UNION ALL
---------
SELECT
original,
substr(remaining, instr(remaining, '.') + 1),
padded || '.' || substr('000000' || remaining, instr(remaining, '.'), 6)
FROM
split_str
WHERE
remaining != ''
)
SELECT
original,
padded
FROM
split_str
WHERE
remaining = ''
ORDER BY
padded
Demo: db<>fiddle.uk
(You may or may not want to strip the leading ., depending on your needs.)

How to Extract the middle characters from a string without an specific index to start and end it before the space character is read?

I have a column that contains data like:
AB-123 XYZ
ABCD-456 AAA
BCD-789 BBB
ZZZ-963
Y-85
and this is what i need from those string:
123
456
789
963
85
I need the characters from the left after the dash('-') character, then ends before the space character is read.
Thank You guys.
Note: Original tag on this question was Oracle and this answer is based on that tag. Now that, tag is updated to SqlServer, this answer is no longer valid, if somebody looking for Oracle solution, this may help.
Use regular expression to arrive at sub string.
select trim(substr(regexp_substr('ABCD-456 AAA','-[0-9]+ '),2)) from dual
'-[0-9]+ ' will grab any string pattern which starts with dash has one or more digits and ends with a ' ' and returns number with dash
substr will remove '-' from above output
trim will remove any trailing ' '
Check This.
Using Substring and PatIndex.
select
SUBSTRING(colnm, PATINDEX('%[0-9]%',colnm),
PATINDEX('%[^0-9]%',ltrim(RIGHT(colnm,LEN(colnm)-CHARINDEX('-',colnm)))))
from
(
select 'AB-123 XYZ' colnm union
select 'ABCD-456 AAA' union
select 'BCD-789 BBB' union
select 'ZX- 23 BBB'
)a
OutPut :
Try this
http://rextester.com/YTBPQD69134
CREATE TABLE Table1 ([col] varchar(12));
INSERT INTO Table1
([col])
VALUES
('AB-123 XYZ'),
('ABCD-456 AAA'),
('BCD-789 BBB');
select substring
(col,
charindex('-',col,1)+1,
charindex(' ',col,1)-charindex('-',col,1)
) from table1;
Assume all values has '-' and followed by a space ' '. Below solution will not tolerant to exception case:
SELECT
*,
SUBSTRING(Value, StartingIndex, Length) AS Result
FROM
-- You can replace this section of code with your table name
(VALUES
('AB-123 XYZ'),
('ABCD-456 AAA'),
('BCD-789 BBB')
) t(Value)
-- Use APPLY instead of sub-query is for debugging,
-- you can view the actual parameters in the select
CROSS APPLY
(
SELECT
-- Get the first index of character '-'
CHARINDEX('-', Value) + 1 AS StartingIndex,
-- Get the first index of character ' ', then calculate the length
CHARINDEX(' ', Value) - CHARINDEX('-', Value) - 1 AS Length
) b