NOT IN is not working as expected with Listagg function - sql

Below is the DDL of the table
create or replace table tempdw.blk_table;
(
db_name varchar,
tbl_expr varchar
);
insert into tempdw.blk_table values ('edw','ABC%');
insert into tempdw.blk_table values ('edw','EFG%');
select * from tempdw.blk_table;
Below code is not working, expected output should not return any
select * from tempdw.blk_table where tbl_expr not in (
select regexp_replace(regexp_replace(replace(listagg(tbl_expr,','),',','\',\''),'^','\''),'$','\'') from tempdw.blk_table);
When I run below code it works fine , Trying to understand why it's not working for above code
select * from tempdw.blk_table where tbl_expr NOT IN('ABC%','EFG%');

Au contraire The code is working just fine. You don't understand the difference between a string that has commas and a list of strings.
Unfortunately, it is rather hard to figure out what you do want to do, because your question does not explain that.
I can speculate that you want something like:
select bt.*
from blk_table bt
where db_name like tbl_expr;
This is just a guess, however.

with data as (
select * from values ('edw','ABC%'),('edw','ABC%') v(db_name,tbl_expr )
)
select * from data
where tbl_expr not in (
select regexp_replace(regexp_replace(replace(listagg(tbl_expr,','),',','\',\''),'^','\''),'$','\'') from data);
does indeed give the results you don't want. aka:
DB_NAME TBL_EXPR
edw ABC%
edw ABC%
because your sub-query only has one row of results, because you have aggregated the two input into one row.
REGEXP_REPLACE( REGEXP_REPLACE( REPLACE( LISTAGG( TBL_EXPR,','),',','\',\''),'^','\''),'$','\'')
'ABC%','ABC%'
and NOT IN is a exact match .. thus if we change from strings to numbers:
SELECT num, num in (2,3,4) FROM values (1),(3),(5) v(num);
gives:
NUM NUM IN (2,3,4)
1 0
3 1
5 0
so your NOT IN would only return strings that are not in the list of one you have... and given your list is the aggregate of the same input, that are by definition not that same.
back to strings..
SELECT str
,str in ('str_a', 'str_b')
,str not in ('str_a', 'str_b')
from values ('a'),('str_b') v(str);
gives:
STR STR IN ('STR_A', 'STR_B') STR NOT IN ('STR_A', 'STR_B')
a 0 1
str_b 1 0
Thus the results you are getting..
now I suspect you are want LIKE type behavior OR a REGEX match, but given you are building the list you know what you are doing there..
also note:
listagg(tbl_expr,',') AS a
,replace(a,',','\',\'') AS b
,regexp_replace(b,'^','\'') AS c
,regexp_replace(c,'$','\'') AS d
is the effect of what you are doing can be replaced with
listagg('\'' || tbl_expr || '\'',',')
unless you want strings with embedded comma to become independent "list" items..

Related

Capturing particular part of Integer Value from part of a String value

I have a table like cust_attbr consists column attbr which has values like:
{"SRCTAXAMT":"11300",เอ็ก10110","TAXAMT":"11300","LOGID":"190301863","VAT_NUMBER":"0835546003122"}
{"SRCTAXAMT":"11300", กรุงค10110","TAXAMT":"11300","LOGID":"190301863","VAT_NUMBER":"0835546003122"}
........ ... ...
{"SRCTAXAMT":"11300", กรุงค10110","TAXAMT":"11300","LOGID":"190301863","VAT_NUMBER":" "}
So, I have to write one select statement which will fetch only VAT_NUMBER value like:
0835546003122
0835546003122
.... ... ..
null
With sample data you posted:
SQL> select * From test;
ID ATTBR
---------- ----------------------------------------------------------------------------------------------------------------
1 "{"SRCTAXAMT":"11300",????10110","TAXAMT":"11300","LOGID":"190301863","VAT_NUMBER":"0835546003122"}"
2 "{"SRCTAXAMT":"11300", ?????10110","TAXAMT":"11300","LOGID":"190301863","VAT_NUMBER":"0835546003122"}"
3 "{"SRCTAXAMT":"11300", ?????10110","TAXAMT":"11300","LOGID":"190301863","VAT_NUMBER":" "}"
this might be one option:
SQL> select id,
2 regexp_substr(regexp_substr(attbr, 'VAT_NUMBER":"(\d+)?'), '\d+$') vat
3 from test;
ID VAT
---------- --------------------
1 0835546003122
2 0835546003122
3
SQL>
Inner regexp_substr returns VAT_NUMBER followed by optional number, while the outer one extracts only the number anchored to the end of the previous substring.
If you're on 18c and the data is actual json (it currently is not because of the double quotes around the curly braces and the ",.กรุงค10110" - It is unclear that this is because of your sample data) you could use json_table function:
WITH t (json_val) AS
(
SELECT '{"SRCTAXAMT":"11300","TAXAMT":"11300","LOGID":"190301863","VAT_NUMBER":"0835546003122"}' FROM DUAL UNION ALL
SELECT '{"SRCTAXAMT":"11300","TAXAMT":"11300","LOGID":"190301863","VAT_NUMBER":"0835546003122"}' FROM DUAL UNION ALL
SELECT '{"SRCTAXAMT":"11300","TAXAMT":"11300","LOGID":"190301863","VAT_NUMBER":" "}' FROM DUAL
)
SELECT jt.*
FROM t,
JSON_TABLE(json_val, '$'
COLUMNS (first_name VARCHAR2(50 CHAR) PATH '$."VAT_NUMBER"')) jt;
0835546003122
0835546003122
One option would be converting those column values to JSON syntax an then extract the values of VAT_NUMBER keys provided DB version is 12c Release 1+. Here, we have an issue that there are unrecognized characters, probably an alphabet from far east and those strings are not properly quoted, then we need to remove the part upto TAXAMT key, and then extracting VAT_NUMBER key's value through prefixing an opening curly brace('{') by use of JSON_VALUE() function :
SELECT JSON_VALUE(
'{'||REGEXP_REPLACE(str,'(.*10110",)(.*)','\2'),
'$.VAT_NUMBER'
) AS VAT_NUMBER
FROM tab --> your original data source
Demo

Sybase LIST function truncating results after 256 characters

I'm trying to aggregate a column of strings into one cell by concatenating them together and separating them with commas using syabase's LIST fuction. But the results get truncated after 256 characters. Does anyone know of a way to fix this or if there are any alternatives that would give me what I'm looking for.
For example, if I have a table myTable that looks like this:
myVal
-------------
'0000000001'
'0000000002'
'0000000003'
'0000000004'
'0000000005'
'0000000006'
'0000000007'
'0000000008'
'0000000009'
'0000000010'
'0000000011'
'0000000012'
'0000000013'
'0000000014'
'0000000015'
'0000000016'
'0000000017'
'0000000018'
'0000000019'
'0000000020'
'0000000021'
'0000000022'
'0000000023'
'0000000024'
'0000000025'
'0000000026'
'0000000027'
'0000000028'
'0000000029'
'0000000030'
then run the following query:
select list(myVal,',') as myResult from myTable
I get the following result
myResult
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0000000001,0000000002,0000000003,0000000004,0000000005,0000000006,0000000007,0000000008,0000000009,0000000010,0000000011,0000000012,0000000013,0000000014,0000000015,0000000016,0000000017,0000000018,0000000019,0000000020,0000000021,0000000022,0000000023,000
Notice the result string gets truncated after 0000000023
Try this:
SELECT
LIST( CAST( myVal AS nvarchar(max) ), ',' )
FROM
myTable

Escaping single quotes in REDSHIFT SQL

I've lots of string values containing single quotes which I need to insert to a column in REDSHIFT table.
I used both /' and '' to escape the single quote in INSERT statement.
e.g.
INSERT INTO table_Temp
VALUES ('1234', 'O\'Niel'), ('3456', 'O\'Brien')
I also used '' instead of \' but it keeps giving me error that "VALUES list must of same length" i.e. no: of arguments for each record >2.
Can you let know how to have this issue resolved?
The standard in SQL is double single quotes:
INSERT INTO table_Temp (col1, col2) -- include the column names
VALUES ('1234', 'O''Niel'), ('3456', 'O''Brien');
You should also include the column names corresponding to the values being inserted. That is probably the cause of your second error.
You could use CHR(39) and concat the strings. Your name would look like below:
('O' || CHR(39)||'Brian')
I think it may depend on your environment. I'm using Periscope Data's redshift SQL editor, and \ worked as an escape character. '' and \\ did not work.
I was facing similar problem , I was needing send a kind of JSON structure to then decode it into my query but there was a program receiving my string and this program was escaping my escapes, so the query fails, finally I found this :
Put $$ in dollar-quoted string in PostgreSQL
mentioning quote_literal(42.5)
https://www.postgresql.org/docs/current/functions-string.html#FUNCTIONS-STRING-OTHER
This resolves my issue . an example
String is
'LocalTime={US/Central}; NumDays={1}; NumRows={3}; F_ID={[Apple, Orange, Bannana]}'
Select
Param, value , replace(quote_literal(replace(replace(Value,'[',''),']','')),',',quote_literal(',')) ValueList
FROM (
select
SPLIT_PART(split,'=',1) as Param,
replace( replace(SPLIT_PART(split,'=',2),'{',''),'}','') as Value
FROM
(
select
trim(split_part(freeform.txt, ';', number.n)) as split
from
( select
'LocalTime={US/Central}; NumDays={1}; NumRows={3}; F_ID={[Apple, Orange, Bannana]}' as txt
) freeform,
( select 1 as n union all
select 2 union all
select 3 union all
select 4 union all
select 5 union all
select 6 union all
select 7 union all
select 8 union all
select 9 union all
select 10
) number
where split <> ''
) as MY_TMP
) as valuePart
use \\' to escape '
s = s.replace("'", "\\'")

to_number from char sql

I have to select only the IDs which have only even digits (an ID looks like: p19 ,p20 etc). That is, p20 is good (both 2 and 0 are even digits); p18 is not.
I thought to use substr to get each number from the IDs and then see if it's even .
select from profs
where to_number(substr(id_prof,2,2))%2=0 and to_number(substr(id_prof,3,2))%2=0;
IF you need all rows consist of 'p' in beginning and even digits on tail It should look like:
select *
from profs
where regexp_like (id_prof, '^p[24680]+$');
with
profs ( prof_id ) as (
select 'p18' from dual union all
select 'p24' from dual union all
select 'p53' from dual
)
-- End of test data; what is above this line is NOT part of the solution.
-- The solution (SQL query) begins here.
select *
from profs
where length(prof_id) = length(translate(prof_id, '013579', '0'));
PROF_ID
-------
p24
This solution should work faster than anything using regular expressions. All it does is to replace 0 with itself and DELETE all odd digits from the input string. (The '0' is included due to a strange but documented behavior of translate() - the third argument can't be empty). If the length of the input string doesn't change after the translation, that means the input string didn't have any odd digits.
where mod(to_number(regexp_replace(id_prof, '[^[:digit:]]', '')),2) = 0

Is it possible to query a comma separated column for a specific value?

I have (and don't own, so I can't change) a table with a layout similar to this.
ID | CATEGORIES
---------------
1 | c1
2 | c2,c3
3 | c3,c2
4 | c3
5 | c4,c8,c5,c100
I need to return the rows that contain a specific category id. I starting by writing the queries with LIKE statements, because the values can be anywhere in the string
SELECT id FROM table WHERE categories LIKE '%c2%';
Would return rows 2 and 3
SELECT id FROM table WHERE categories LIKE '%c3%' and categories LIKE '%c2%'; Would again get me rows 2 and 3, but not row 4
SELECT id FROM table WHERE categories LIKE '%c3%' or categories LIKE '%c2%'; Would again get me rows 2, 3, and 4
I don't like all the LIKE statements. I've found FIND_IN_SET() in the Oracle documentation but it doesn't seem to work in 10g. I get the following error:
ORA-00904: "FIND_IN_SET": invalid identifier
00904. 00000 - "%s: invalid identifier"
when running this query: SELECT id FROM table WHERE FIND_IN_SET('c2', categories); (example from the docs) or this query: SELECT id FROM table WHERE FIND_IN_SET('c2', categories) <> 0; (example from Google)
I would expect it to return rows 2 and 3.
Is there a better way to write these queries instead of using a ton of LIKE statements?
You can, using LIKE. You don't want to match for partial values, so you'll have to include the commas in your search. That also means that you'll have to provide an extra comma to search for values at the beginning or end of your text:
select
*
from
YourTable
where
',' || CommaSeparatedValueColumn || ',' LIKE '%,SearchValue,%'
But this query will be slow, as will all queries using LIKE, especially with a leading wildcard.
And there's always a risk. If there are spaces around the values, or values can contain commas themselves in which case they are surrounded by quotes (like in csv files), this query won't work and you'll have to add even more logic, slowing down your query even more.
A better solution would be to add a child table for these categories. Or rather even a separate table for the catagories, and a table that cross links them to YourTable.
You can write a PIPELINED table function which return a 1 column table. Each row is a value from the comma separated string. Use something like this to pop a string from the list and put it as a row into the table:
PIPE ROW(ltrim(rtrim(substr(l_list, 1, l_idx - 1),' '),' '));
Usage:
SELECT * FROM MyTable
WHERE 'c2' IN TABLE(Util_Pkg.split_string(categories));
See more here: Oracle docs
Yes and No...
"Yes":
Normalize the data (strongly recommended) - i.e. split the categorie column so that you have each categorie in a separate... then you can just query it in a normal faschion...
"No":
As long as you keep this "pseudo-structure" there will be several issues (performance and others) and you will have to do something similar to:
SELECT * FROM MyTable WHERE categories LIKE 'c2,%' OR categories = 'c2' OR categories LIKE '%,c2,%' OR categories LIKE '%,c2'
IF you absolutely must you could define a function which is named FIND_IN_SET like the following:
CREATE OR REPLACE Function FIND_IN_SET
( vSET IN varchar2, vToFind IN VARCHAR2 )
RETURN number
IS
rRESULT number;
BEGIN
rRESULT := -1;
SELECT COUNT(*) INTO rRESULT FROM DUAL WHERE vSET LIKE ( vToFine || ',%' ) OR vSET = vToFind OR vSET LIKE ('%,' || vToFind || ',%') OR vSET LIKE ('%,' || vToFind);
RETURN rRESULT;
END;
You can then use that function like:
SELECT * FROM MyTable WHERE FIND_IN_SET (categories, 'c2' ) > 0;
For the sake of future searchers, don't forget the regular expression way:
with tbl as (
select 1 ID, 'c1' CATEGORIES from dual
union
select 2 ID, 'c2,c3' CATEGORIES from dual
union
select 3 ID, 'c3,c2' CATEGORIES from dual
union
select 4 ID, 'c3' CATEGORIES from dual
union
select 5 ID, 'c4,c8,c5,c100' CATEGORIES from dual
)
select *
from tbl
where regexp_like(CATEGORIES, '(^|\W)c3(\W|$)');
ID CATEGORIES
---------- -------------
2 c2,c3
3 c3,c2
4 c3
This matches on a word boundary, so even if the comma was followed by a space it would still work. If you want to be more strict and match only where a comma separates values, replace the '\W' with a comma. At any rate, read the regular expression as:
match a group of either the beginning of the line or a word boundary, followed by the target search value, followed by a group of either a word boundary or the end of the line.
As long as the comma-delimited list is 512 characters or less, you can also use a regular expression in this instance (Oracle's regular expression functions, e.g., REGEXP_LIKE(), are limited to 512 characters):
SELECT id, categories
FROM mytable
WHERE REGEXP_LIKE('c2', '^(' || REPLACE(categories, ',', '|') || ')$', 'i');
In the above I'm replacing the commas with the regular expression alternation operator |. If your list of delimited values is already |-delimited, so much the better.