SQL Search rows that contain strings from 2nd table list - sql

I have a master table that contains a list of strings to search for. it returns TRUE/FALSE if any string in the cell contains text from the master lookup table. Currently I use excel's
=SUMPRODUCT(--ISNUMBER(SEARCH(masterTable,[#searchString])))>0
is there a way to do something like this in SQL? LEFT JOIN or OUTER APPLY would be simple solutions if the strings were equal; but they need be contains..
SELECT *
FROM t
WHERE col1 contains(lookupString,lookupColumn)
--that 2nd table could be maintained and referenced from multiple queries
hop
bell
PRS
2017
My desired results would be a column that shows TRUE/FALSE if the row contains any string from the lookup table
SEARCH_STRING Contained_in_lookup_column
hopping TRUE
root FALSE
Job2017 TRUE
PRS_tool TRUE
hand FALSE

Sorry i dont have access to the DB now to confirm the syntax, but should be something like this:
SELECT t.name,
case when (select count(1) from data_table where data_col like '%' || t.name || '%' > 0) then 'TRUE' else 'FALSE' end
FROM t;
or
SELECT t.name,
case when exists(select null from data_table where data_col like '%' || t.name || '%') then 'TRUE' else 'FALSE' end
FROM t;
Sérgio

You can use a combination of % wildcards with LIKE and EXISTS.
Example (using Oracle syntax) - we have a v_data table containing the data and a v_queries table containing the query terms:
with v_data (pk, value) as (
select 1, 'The quick brown fox jumps over the lazy dog' from dual union all
select 2, 'Yabba dabba doo' from dual union all
select 3, 'forty-two' from dual
),
v_queries (text) as (
select 'quick' from dual union all
select 'forty' from dual
)
select * from v_data d
where exists (
select null
from v_queries q
where d.value like '%' || q.text || '%');

Related

In BigQuery, identify when columns do not match on UNION ALL

with
table1 as (
select 'joe' as name, 17 as age, 25 as speed
),
table2 as (
select 'nick' as name, 21 as speed, 23 as strength
)
select * from table1
union all
select * from table2
In Google BigQuery, this union all does not throw an error because both tables have the same number of columns (3 each). However I receive bad data output because the columns do not match. Rather than outputting a new table with 4 columns name, age, speed, strength with correct values + nulls for missing values (which would probably be preferred), the union all keeps the 3 columns from the top row.
Is there a good way to catch that the columns do not match, rather than the query silently returning bad data? Is there any way for this to return an error perhaps, as opposed to a successful table? I'm not sure how to check in SQL that the columns in the 2 tables match.
Edit: in this example it is clear to see that the columns do not match, however in our data we have 100+ columns and we want to avoid a situation where we make an error in a UNION ALL
Below is for BigQuery Standard SQL and using scripting feature of BQ
DECLARE statement STRING;
SET statement = (
WITH table1_columns AS (
SELECT column FROM (SELECT * FROM `project.dataset.table1` LIMIT 1) t,
UNNEST(REGEXP_EXTRACT_ALL(TRIM(TO_JSON_STRING(t), '{}'), r'"([^"]*)":')) column
), table2_columns AS (
SELECT column FROM (SELECT * FROM `project.dataset.table2` LIMIT 1) t,
UNNEST(REGEXP_EXTRACT_ALL(TRIM(TO_JSON_STRING(t), '{}'), r'"([^"]*)":')) column
), all_columns AS (
SELECT column FROM table1_columns UNION DISTINCT SELECT column FROM table2_columns
)
SELECT (
SELECT 'SELECT ' || STRING_AGG(IF(t.column IS NULL, 'NULL as ', '') || a.column, ', ') || ' FROM `project.dataset.table1` UNION ALL '
FROM all_columns a LEFT JOIN table1_columns t USING(column)
) || (
SELECT 'SELECT ' || STRING_AGG(IF(t.column IS NULL, 'NULL as ', '') || a.column, ', ') || ' FROM `project.dataset.table2`'
FROM all_columns a LEFT JOIN table2_columns t USING(column)
)
);
EXECUTE IMMEDIATE statement;
when applied to sample data from your question - output is
Row name age speed strength
1 joe 17 25 null
2 nick null 21 23
After saving table1 and table2 as 2 tables in a dataset in BigQuery, I then used the metadata using INFORMATION_SCHEMA to check that the columns matched.
SELECT *
FROM models.INFORMATION_SCHEMA.COLUMNS
where table_name = 'table1'
SELECT *
FROM models.INFORMATION_SCHEMA.COLUMNS
where table_name = 'table2'
INFORMATION_SCHEMA.COLUMNS returns information including the column names and their positioning. I can join these 2 tables together then to check that the names match...

How to extract this specific data from a particular column in SQL Server?

I have column with below data:
Change
18 MCO-005329
A ECO-12239
0 ECO-25126
X1 ECO-05963
NA MCO-003778
C ECO-08399
MCO-003759
ECO-00643217
NULL
I want to extract the output like below:
MCO-005329
ECO-12239
ECO-25126
ECO-05963
MCO-003778
ECO-08399
MCO-003759
ECO-00643217
I have implemented the code like below:
select DISTINCT change,
case when change like 'MCO%' THEN change when change like 'ECO-%' THEN change
when change like '%MCO-%' then LTRIM(RTRIM(SUBSTRING(change,10,19) ))
when change like '%ECO-%' then LTRIM(RTRIM(SUBSTRING(change,10,19) ))
else '' end x from table
You can parse out the values from your requirements using SPLIT_STRING, outer apply, and a simple where clause without relying on hard coding any specific string length or position values, its dynamic.
SELECT D2.*
FROM
(
select '18 MCO-005329'
union select 'A ECO-12239'
union select '0 ECO-25126'
union select 'X1 ECO-05963'
union select 'NA MCO-003778'
union select 'C ECO-08399'
union select 'MCO-003759'
union select 'ECO-00643217'
union select NULL
) T(Change)
outer apply
(
select value
from
string_split(Change, ' ') d
) d2
where d2.value like '%-%' or d2.value is null
If you dont want nulls then smiply remove or d2.value is null
https://learn.microsoft.com/en-us/sql/t-sql/queries/from-transact-sql?view=sql-server-ver15
https://learn.microsoft.com/en-us/sql/t-sql/functions/string-split-transact-sql?view=sql-server-ver15
You could use CHARINDEX() and RIGHT() as
SELECT *, RIGHT(Change, CHARINDEX('-', REVERSE(Change)) + 3)
FROM
(
VALUES
('18 MCO-005329'),
('A ECO-12239'),
('0 ECO-25126'),
('X1 ECO-05963'),
('NA MCO-003778'),
('C ECO-08399'),
('MCO-003759'),
('ECO-00643217'), ('hhh kkk-k'),
(NULL)
) T(Change)

Select Value Match - SQL

Can you advise if it is possible, to select a count for numerous substrings in a query
so if I have a message field which contains for example, text messages and I could do
SELECT COUNT(1)
FROM MESSAGES
WHERE MESSAGE_BODY LIKE '%hello%'
but what I want to do is more:
SELECT STRING, COUNT(1)
FROM MESSAGES
WHERE MESSAGE_BODY IN (list of strings with wild card)
is this possible?
to break down example:
ID | Message_Body
1 | Hello, How Are You?
2 | Hi, Great Thanks
3 | Hello, How is things?
4 | Ciao
Output wanted:
hello , 2
ciao, 1
SELECT (input strings), COUNT(1)
FROM TABLE
WHERE (input strings) IN ('%hello%','%ciao%')
If I understood you correctly, you can try something like this:
SELECT t.string,
CASE WHEN t.MESSAGE_BODY LIKE '%laptop%' then 1 else 0 END +
CASE WHEN t.MESSAGE_BODY LIKE '%one%' then 1 else 0 END +
CASE WHEN t.MESSAGE_BODY LIKE '%two%' then 1 else 0 END as count_col
FROM YourTable t
If you just want multiple LIKE comaparison, use REGEXP_LIKE() :
SELECT STRING, COUNT(1)
FROM MESSAGES
where regexp_like(MESSAGE_BODY, 'one|two|laptop')
EDIT: You can use a derived table containing all strings you are intrested on and left join to the original table for count:
SELECT t.wrd,COUNT(s.id) as cnt
FROM (
SELECT 'hello' as wrd FROM DUAL
UNION ALL
SELECT 'ciao' as wrd FROM DUAL) t
LEFT OUTER JOIN messages s
ON(s.message_body LIKE '%' || t.wrd || '%')
GROUP BY t.wrd
Here is with looking for whole words:
SELECT a.word, COUNT (message.message_body)
FROM ( SELECT REGEXP_SUBSTR ('hello,ciao', '[^,]+', 1, LEVEL) word
FROM DUAL
CONNECT BY REGEXP_SUBSTR ('hello,ciao', '[^,]+', 1, LEVEL) IS NOT NULL) a
LEFT OUTER JOIN MESSAGES ON REGEXP_INSTR (MESSAGE_BODY, '(^|\s)' || a.word || '(\s|$)', 1, 1, 0, 'i') > 0
GROUP BY a.word

Like Operator for checking multiple words

I'm struggling for a like operator which works for below example
Words could be
MS004 -- GTER
MS006 -- ATLT
MS009 -- STRR
MS014 -- GTEE
MS015 -- ATLT
What would be the like operator in Sql Server for pulling data which will contain words like ms004 and ATLT or any other combination like above.
I tried using multiple like for example
where column like '%ms004 | atl%'
but it didn't work.
EDIT
Result should be combination of both words only.
Seems you are looking for this.
`where column like '%ms004%' or column like '%atl%'`
or this
`where column like '%ms004%atl%'
;WITH LikeCond1 as (
SELECT 'MS004' as L1 UNION
SELECT 'MS006' UNION
SELECT 'MS009' UNION
SELECT 'MS014' UNION
SELECT 'MS015')
, LikeCond2 as (
SELECT 'GTER' as L2 UNION
SELECT 'ATLT' UNION
SELECT 'STRR' UNION
SELECT 'GTEE' UNION
SELECT 'ATLT'
)
SELECT TableName.*
FROM LikeCond1
CROSS JOIN LikeCond2
INNER JOIN TableName ON TableName.Column like '%' + LikeCond1.L1 + '%'
AND TableName.Column like '%' + LikeCond2.L2 + '%'
Try like this
select .....from table where columnname like '%ms004%' or columnname like '%atl%'

SQL Server dynamic column list in contains function

I have a database table in SQL Server 2008 with 5 nvarchar(max) columns. We're using the CONTAINS function to look for text in these columns.
We can look in all five columns using this kind of query:
SELECT *
FROM SomeTable ST
WHERE CONTAINS( (ST.ColumnA, ST.ColumnB, ST.ColumnC, ST.ColumnD, ST.ColumnE) , '"Strawberry"')
Now we want to search for text for one or more of these columns dynamically. For example, only look in ColumnB and ColumnE. I tried using a CASE statement, but I couldn't.
The only solution I can think of is dynamic SQL, but I'd prefer to avoid this. (The full query is very complicated.) Is there a way to do this without using dynamic SQL?
The only way I can think of to avoid using dynamic SQL involves using a temp table and if statements and stored proc. Have a stored proc with parameters for each of the columns that include the text to search that columns for. Create a temp table or table variable to store interim results.
Have five different if statements one for each possible column. In each if insert to the temp table if the variable is not null. something like:
IF #ColA is not null
BEGIN
INSERT INTO #temp
SELECT * FROM SomeTable ST
WHERE CONTAINS( (ST.ColumnA) , #ColA)
END
At the end select from the temp table to show your result.
This only works well though if you don't have very many columns and it would be unlikely that more will be added. Frankly, the fact that you have multiple columns you need to do full text search for the same search string indicates to me that you may have a basic problem with the database design.
I just can think in a solution like this, but this doesn't use full text Search... let me know if it is useful for you.
SELECT *
FROM SomeTable ST
WHERE 1=1
AND ((#colA = '') OR ST.ColumnA LIKE #colA)
AND ((#colB = '') OR ST.ColumnB LIKE #colB)
AND ((#colC = '') OR ST.ColumnC LIKE #colC)
AND ((#colD = '') OR ST.ColumnD LIKE #colD)
AND ((#colE = '') OR ST.ColumnE LIKE #colE)
The search_on parameters would be bit (0, 1) so, when set on 1, the search on that column would be activated.
You could UNION the 5 individual cases, then PIVOT and concatenate. I'm not sure it's any better than dynamic SQL.
You would end up with something like:
SET #FindKey = '%B%E%' -- This is your search which field criteria
WITH RESULTS1 AS (
SELECT PK, 'A' AS Col
FROM SomeTable ST
WHERE #FindKey LIKE '%A%' AND CONTAINS(ST.ColumnA, '"Strawberry"')
UNION
SELECT PK, 'B' AS Col
FROM SomeTable ST
WHERE #FindKey LIKE '%B%' AND CONTAINS(ST.ColumnB, '"Strawberry"')
UNION
SELECT PK, 'C' AS Col
FROM SomeTable ST
WHERE #FindKey LIKE '%C%' AND CONTAINS(ST.ColumnC, '"Strawberry"')
UNION
SELECT PK, 'D' AS Col
FROM SomeTable ST
WHERE #FindKey LIKE '%D%' AND CONTAINS(ST.ColumnD, '"Strawberry"')
UNION
SELECT PK, 'E' AS Col
FROM SomeTable ST
WHERE #FindKey LIKE '%E%' AND CONTAINS(ST.ColumnE, '"Strawberry"')
)
,RESULTS2 AS (
SELECT PK, ISNULL([A], '') + ISNULL([B], '') + ISNULL([C], '') + ISNULL([D], '') + ISNULL([E], '') AS FoundKey
FROM RESULTS PIVOT ( MIN(Col) FOR Col IN ([A], [B], [C], [D], [E]) ) AS pvt
)
SELECT *
FROM SomeTable
INNER JOIN RESULTS2
ON RESULTS2.PK = SomeTable.PK
WHERE RESULTS2.FoundKey LIKE #FindKey