Oracle SQL Repeated words in the String - sql

Question

I need your suggestions/inputs on of the following task. I have the following table:
ID ID_NAME
------ ---------------------------------
1 TOM HANKS TOM JR
2 PETER PATROL PETER JOHN PETER
3 SAM LIVING
4 JOHNSON & JOHNSON INC
5 DUHGT LLC
6 THE POST OF THE OFFICE
7 TURNING REP WEST
8 GEORGE JOHN
I Need a SQL query to find a repetitive word for every ID. if it exists, i need to get the count of the repeated word.
For instance in ID 2, the word PETER was repeated 3 times and in ID 1 the word TOM was repeated twice. so I need the output something like this:
ID ID_NAME COUNT
------ --------------------------------- --------
1 TOM HANKS TOM JR 2
2 PETER PATROL PETER JOHN PETER 3
3 SAM LIVING 0
4 JOHNSON & JOHNSON INC 2
5 DUHGT LLC 0
6 THE POST OF THE OFFICE 2
7 TURNING REP WEST 0
8 GEORGE JOHN 0
Just an FYI, The table has 560K rows
I tried the below and it didn't work and it is literally looking for every single word.
SELECT RESULT, COUNT(*)
FROM (SELECT
REGEXP_SUBSTR(COL_NAME, '[^ ]+', 1, COLUMN_VALUE) RESULT
FROM TABLE_NAME T ,
TABLE(CAST(MULTISET(SELECT DISTINCT LEVEL
FROM TABLE_NAME X
CONNECT BY LEVEL <= LENGTH(X.COL_NAME) - LENGTH(REPLACE(X.COL_NAME, ' ', '')) + 1
) AS SYS.ODCINUMBERLIST)) T1
)
WHERE RESULT IS NOT NULL
GROUP BY RESULT
ORDER BY 1;
Please let me know your inputs.

Answer 1

The query below counts repeated words and returns the highest count (if a word appears three times and another appears twice, the result will be the number 3). It treats JOHN as different from John (if capitalization shouldn't count as "different" then wrap the input strings within UPPER(...)). It only considers space as a word delimiter; if something else, like dash, is also considered as a delimiter, add to the REGEXP search pattern. Make sure you put a dash right at the end of a square-bracketed matching character list, etc. - the usual "tricks" for matching character lists. More generally, adapt as needed.
The query first breaks each input string into individual words, and counts how many times each word appears. For the count, I only need the words ("tokens") in the GROUP BY clause, I don't need to actually SELECT them, this is why the innermost query may look odd if you aren't forewarned. (Now you are!)
It also seems you want to show null rather than 1 if there are no repeated words, so I wrote the query to accommodate that. (Not sure why 1 wasn't OK.)
with
test_data ( id, id_name ) as (
select 1, 'TOM HANKS TOM JR' from dual union all
select 2, 'PETER PATROL PETER JOHN PETER' from dual union all
select 3, 'SAM LIVING' from dual union all
select 4, 'JOHNSON & JOHNSON INC' from dual union all
select 5, 'DUHGT LLC' from dual union all
select 6, 'THE POST OF THE OFFICE' from dual union all
select 7, 'TURNING REP WEST' from dual union all
select 8, 'GEORGE JOHN' from dual
)
-- end of test data; SQL query begins below this line
select id, id_name, case when max(cnt) >= 2 then max(cnt) end as max_count
from (
select id, id_name, count(*) as cnt
from test_data
connect by level <= 1 + regexp_count(id_name, ' ')
and prior id = id
and prior sys_guid() is not null
group by id, id_name, regexp_substr(id_name, '[^ ]+', 1, level)
)
group by id, id_name
order by id -- if needed
;
Output:
ID ID_NAME MAX_COUNT
-- ----------------------------- ----------
1 TOM HANKS TOM JR 2
2 PETER PATROL PETER JOHN PETER 3
3 SAM LIVING
4 JOHNSON & JOHNSON INC 2
5 DUHGT LLC
6 THE POST OF THE OFFICE 2
7 TURNING REP WEST
8 GEORGE JOHN
8 rows selected.
EDIT:
If you only need to find the returns where the string column has at least one repeated word, and you don't care what the highest "repeated word count" is or how many words are repeated, the solution is simpler and more efficient; you don't need to split the input string into component words and count them.
(The OP indicated in the comments, after long dialogue, that this would suffice.)
In the solution the "match pattern" in regexp_like searches for a string of letters, preceded by either the beginning of the string or a space or a dash and ended by space, comma, period, question mark, exclamation point or dash. Both "markers", for beginning and end of a word, can be modified as needed. Make sure the dash is either the first or last character in [...], anywhere else it has a special meaning.
Then it looks for a repetition of the word. That's what \2 does in the match pattern. It's 2 and not 1 because the "word" is in the second pair of parentheses; I need the first pair for the alternation, EITHER start-of-string OR (space or dash).
Look at the first and the last string for special situations that this query covers correctly. Think of any other possible situations that the query may or may not cover.
with
test_data ( id, id_name ) as (
select 1, 'TOM HANKS TOM-ALAN' from dual union all
select 2, 'PETER PATROL PETER JOHN PETER' from dual union all
select 3, 'SAM LIVING' from dual union all
select 4, 'JOHNSON & JOHNSON INC' from dual union all
select 5, 'DUHGT LLC' from dual union all
select 6, 'THE POST OF THE OFFICE' from dual union all
select 7, 'TURNING REP WEST' from dual union all
select 8, 'GEORGE JOHN-JOHN' from dual
)
-- end of test data; SQL query begins below this line
select id, id_name
from test_data
where regexp_like(id_name, '(^|[ -])([[:alpha:]]+)[ ,.?!-].*\2')
order by id -- if needed
;
ID ID_NAME
-- -----------------------------
1 TOM HANKS TOM-ALAN
2 PETER PATROL PETER JOHN PETER
4 JOHNSON & JOHNSON INC
6 THE POST OF THE OFFICE
8 GEORGE JOHN-JOHN

Answer 2

The next solution find first repeted word and in next step find count of repeating. Edit just now to fix extra subword findings
with s (ID, ID_NAME) as (
select 1, 'TOM HANKS TOM JR' from dual union all
select 10, 'TO TOM TOM TOM TOM TO TO TO STOM HANKS TOM TOMMY' from dual union all
select 2, 'PETER PATROL PETER JOHN PETER' from dual union all
select 3, 'SAM LIVING' from dual union all
select 4, 'qwe JOHNSON & JOHNSON INC' from dual union all
select 5, 'DUHGT LLC' from dual union all
select 6, 'THE POST OF THE OFFICE ' from dual union all
select 7, 'TURNING REP WEST ' from dual union all
select 8, 'GEORGE JOHN ' from dual)
select id,
case when r1 = 0 then 0
else regexp_count(id_name, r3)
- regexp_count(id_name, r3||'\w+') -- exlude word with tail
- regexp_count(id_name, '\w+'||r3) -- exclude words with head
+ regexp_count(id_name, '\w+'||r3||'\w+') -- double calc with head and tail
end as rep_count
from (
select
s.*,
regexp_instr(s.id_name, '(^|\s)(\w+)(\s|$)(.*(\2))+') as r1 ,
regexp_replace(s.id_name, '.*?(^|\s)(\w+)(\s)(.*(\s)\2(\s|$))+.*$', '\2') as r3
from s);
the result is
ID REP_COUNT
---------- ----------
1 2
10 4
2 3
3 0
4 2
5 0
6 2
7 0
8 0