SQL - Stripping a string and using it in a condition - sql

Question

So I have a SQL query issue given to me which i'm struggling to resolve:
It currently brings back 6710445 rows but i need to apply further conditions based on a particular string field.
SELECT
Table1.ExampleColumn1 -- (ID)
,Table1.ExampleColumn2
,Table2.ExampleColumn3
,Table2.ExampleColumn4
,Table3.ExampleColumn5
,Table3.ExampleColumn6
,Table1.StringField
FROM [Example Database].[dbo].[Table1] AS Table1
INNER JOIN [Example Database].[dbo].[Table2] AS Table2
ON Example = Example
INNER JOIN [Example Database].[dbo].[Table3] AS Table3
ON Example = Example
WHERE Month BETWEEN 201304 AND 201603
AND (Age < 19)
The above 'Table1.StringField' has the following type codes displayed as a string in each the rows: "||J183,Y752,J374,Y752."
I also have a reference table (Call it 'Ref1') with 514 of these codes displayed individually, which has no other fields in the table whatsoever.
So what i need to be able to do is find rows from the query above which has any of values from the 'Ref1' displayed anywhere within 'Table1.StringField' individual rows, and if not to not include that row in the results set.
I tried to strip down the 'StringField' column of the comma's and "||" but it didn't work as well as i hoped and ended up bringing back over 30M rows.
Any ideas on how to do this? Preferably so it's efficient and doesn't make the user wait 10 minutes just to query it?

Answer 1

Maybe this will get you half way there... I also agree with Sean Lange's comment about not storing delimited data to begin with but I'm assuming the OP already knows this. You can also pivot/unpivot this data to achieve this as well. This is probably the most brute force way of doing sort of what you're looking to do.
--DROP TABLE #Table
--DROP TABLE #Ref
CREATE TABLE #Table (Col VARCHAR(MAX))
CREATE TABLE #Ref (Code VARCHAR(10))
INSERT INTO #Table (Col) VALUES ('A123,B234,C345'),('A123'),('C345')
INSERT INTO #Ref (Code) VALUES ('A123'),('B234')
SELECT * FROM #Table
SELECT * FROM #Ref
SELECT DISTINCT t.Col
FROM #Table t
CROSS APPLY (
SELECT CASE WHEN CHARINDEX(r.Code, t.Col) > 0 THEN 1 ELSE 0 END AS [ItsHere] FROM #Ref r) oa
WHERE oa.ItsHere = 1

Answer 2

What you need to do is join your query to the Ref1 table on Table1.StringField = Ref1.Ref_1_value and then exclude the Table1 rows that don't match any Ref_1_value. Like this:
SELECT
Table1.ExampleColumn1 -- (ID)
,Table1.ExampleColumn2
,Table2.ExampleColumn3
,Table2.ExampleColumn4
,Table3.ExampleColumn5
,Table3.ExampleColumn6
,Table1.StringField
FROM [Example Database].[dbo].[Table1] AS Table1
INNER JOIN [Example Database].[dbo].[Table2] AS Table2
ON Example = Example
INNER JOIN [Example Database].[dbo].[Table3] AS Table3
ON Example = Example
INNER JOIN [Example Database].[dbo].[Ref1] as Ref1
ON Table1.StringField = Ref1.Ref_1_value
WHERE Month BETWEEN 201304 AND 201603
AND (Age < 19)
AND Ref1.Ref_1_value is not null