My data looks like this
ABCD.(123).(456)
789.(DEF) JKL.MNO
(PQ).(RS).(TUV)||(WXYz)
I am looking to pull the string from the RIGHTMOST parentheses brackets. Results would look like this
(456)
(DEF)
(WXYz)
The entire strings and strings within the parentheses can vary. Its some combination of substring and charindex, but i cannot figure it out.
Such a question suggests a problem with the data structure -- that the string actually consists of multiple items. However, sometimes such string processing is necessary.
The following approach should work, assuming that a parenthesized component always exists:
select t.*, colparen
from t cross apply
(values (right(col, charindex('(', reverse(col)) - 1)) v(colr) cross apply
(values (charindex(colr, charindex(')', col) + 1)) v(colparen)
Since you are 2016, you can use String_Split() in concert with a Cross Apply.
Note: Use Outer Apply if there are no observations and you want to display a null value.
Example
Declare #YourTable table (SomeCol varchar(100))
Insert Into #YourTable values
('ABCD.(123).(456)'),
('789.(DEF).JKL.MNO'),
('(PQ).(RS).(TUV).(WXYz)')
Select B.Value
From #YourTable A
Cross Apply (
Select Top 1 *
,RN=Row_Number() over (Order By (Select Null))
From String_Split(A.SomeCol,'.')
Where Value Like '(%)'
Order by 2 Desc
) B
Returns
Value
(456)
(DEF)
(WXYz)
dbFiddle
select REVERSE(substring(reverse('ABCD.(123).(456)'),CHARINDEX(')',reverse('ABCD.(123).(456)')),CHARINDEX('(',reverse('ABCD.(123).(456)'))))
This should get you what you want
Related
I have in a column some Names and then square brackets with some numbers and letters inside.
How can I extract two values between square brackets and get the results in two different columns?
I start from the Column 'NAME' with the value 'XCDRT [20.9 kd]'
--NAME--
XCDRT [20.9 kd]
qwer [12.234 r.t.]
and I would like to get 3 columns with the values in different columns
-- NAME--- NAME 1--- NAME 2---
--XCDRT---- 20.9-------- kd----
--qwer----- 12.234-------- r.t.-----
Is there a function for such a problem?
I tried to split the value but I don't get the results that I need.
As an alternative solution, if you are on a bleeding edge version of the SQL Server data engine, then you make use of STRING_SPLIT and it's (new) ability to return the ordinal position of a value. Then, with some conditional aggregation, you can unpivot the results:
SELECT TRIM(MAX(CASE N.ordinal WHEN 1 THEN N.[value] END)) AS [Name],
TRIM(MAX(CASE N.ordinal WHEN 2 THEN LEFT(N.[value], CHARINDEX(' ',N.[value] + ' ')) END)) AS [Name1],
TRIM(MAX(CASE N.ordinal WHEN 2 THEN NULLIF(STUFF(N.[value], 1, CHARINDEX(' ',N.[value] + ' '),''),'') END)) AS [Name2]
FROM (VALUES('XCDRT [20.9 kd] qwer [12.234 r.t.]'))V([NAME])
CROSS APPLY STRING_SPLIT(V.[NAME],']',1) R
CROSS APPLY STRING_SPLIT(R.[value],'[',1) N
WHERE R.[value] != ''
GROUP BY V.[NAME],
R.ordinal;
The TRIMs and NULLIF are there to "tidy" the values, as you'd have leading whitespace and incase you don't have a value for Name2.
With a bit of JSON and a CROSS APPLY (or two)
Cross Apply B will split/parse the string
Cross Apply C will create JSON to be consumed.
This will also support N groups of 3
Example
Declare #YourTable Table ([Name] varchar(50)) Insert Into #YourTable Values
('XCDRT [20.9 kd] qwer [12.234 r.t.]')
Select [Name] = JSON_VALUE(JS,'$[0]')
,[Name1] = JSON_VALUE(JS,'$[1]')
,[Name2] = JSON_VALUE(JS,'$[2]')
From #YourTable A
Cross Apply string_split([Name],']') B
Cross Apply ( values ('["'+replace(string_escape(trim(replace(B.Value,'[','')),'json'),' ','","')+'"]') ) C(JS)
Where B.value<>''
Results
Name Name1 Name2
XCDRT 20.9 kd
qwer 12.234 r.t.
Please see the TSQL below:
create table #IDs (id varchar(100))
insert into #IDs values ('123')
insert into #IDs values ('456')
insert into #IDs values ('789')
insert into #IDs values ('1010')
create table #Notes (Note varchar(500))
insert into #Notes values ('Here is a note for 123')
insert into #Notes values ('A note for 789 here')
insert into #Notes values ('456 has a note here')
I want to find all the IDs that are referenced in the #Notes table. This works:
select #IDs.id from #IDs inner join #Notes on #Notes.note like '%' + #IDs.id + '%'
However, there are hundreds of thousands of records in both tables and the query does not complete. I was thinking about FreeText searching, but I don't think it can be applied here. A cursor takes too long to run as well (I think it will take over one month). Is there anything else I can try? I am using SQL Server 2019.
The size of the input is only one aspect of the solution.
By splitting the text to tokens you indeed increase the number of records, but in the same time you enable equality join, which can be implemented using Hash Join.
You should get the query results in a few minutes top, basically the time it takes to your system to do a full scan on both tables, plus some processing time.
No need for temp tables.
No need for indexes.
Select id
from #IDS
where id in (select w.value
from #Notes as n
cross apply string_split(n.Note, ' ') as w
)
Fiddle
Per the OP request -
Here is a code that handles more complicated scenario, where an id could contain various characters (as defined by #token_char) and the separators are potentially all other characters
declare #token_char varchar(100) = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
;
with cte_notes as
(
select Note
,replace(translate(Note,#token_char,space(len(#token_char))),' ','') as non_token_char
from #Notes
)
select id
from #IDS
where id in
(
select w.value
from cte_notes as n
cross apply string_split(translate(n.Note,n.non_token_char,space(len(n.non_token_char))),' ') as w
where w.value != ''
)
The Fiddle data sample was altered accordingly, to reflect the change
If you are going to do this search often you may want to explore using a wonderful (if underused) feature of SQL Server called 'Full Text Search.' To quote Microsoft:
A LIKE query against millions of rows of text data can take minutes to
return; whereas a full-text query can take only seconds or less
against the same data, depending on the number of rows that are
returned.'
I have seen searches go from minutes to seconds using this feature.
You would need to create a Full Text Search Catalog and then create indexs on the tables you want to search. It's not hard and will take you a few minutes to learn how to do this.
This is a good starting point:
https://learn.microsoft.com/en-us/sql/relational-databases/search/get-started-with-full-text-search?view=sql-server-ver15
I would apply CTE with string_split to filter out all alphabetic components and then join #ID table with the result of the CTE by id column. The query was tested on a sample of 1 mm rows.
With CTE As (
Select T.value As id
From #Notes Cross Apply String_Split(Note,' ') As T
Where Try_Convert(Int, T.value) Is Not Null
)
Select I.id
From #IDs As I Inner Join CTE On (I.id=CTE.id)
If you just want to extract a numeric value from a string, in this case join is excessive.
Select T.value As id, #Notes.Note
From #Notes Cross Apply String_Split(Note,' ') As T
Where Try_Convert(Int, T.value) Is Not Null And T.value Like '%[0-9]%'
id
Note
123
Here is a note for 123
789
A note for 789 here
456
456 has a note here
No matter what, under the given circumstances, I would use join to filter out those numbers that are not represented in #IDs table.
With CTE As (
Select distinct(id) As id
From #IDs
)
Select T.value As id, #Notes.Note
From #Notes Cross Apply String_Split(Note,' ') As T
Inner Join CTE On (T.value=CTE.id)
Where Try_Convert(Int, T.value) Is Not Null
And T.value Like '%[0-9]%'
If the string contains brackets or parenthesis instead of spaces like this:
"456(this is an id number) has a note here" or "456[01/01/2022]"
as last resorts (since it degrades performance) you can use TRANSLATE to replace those brackets with spaces as follows:
With CTE As (
Select distinct(id) As id
From #IDs
)
Select T.value As id, #Notes.Note
From #Notes Cross Apply String_Split(TRANSLATE(Note,'[]()',' '),' ') As T
Inner Join CTE On (T.value=CTE.id)
Where Try_Convert(Int, T.value) Is Not Null
And T.value Like '%[0-9]%'
db<>fiddle
CREATE TABLE #StudentClasses
(
ID INT,
Student VARCHAR(100),
Classes VARCHAR(100),
CCode VARCHAR(30)
)
GO
INSERT INTO #StudentClasses
SELECT 1, 'Mark', 'Maths,Science,English', 'US,UK,AUS'
UNION ALL
SELECT 2, 'John', 'Science,English', 'BE,DE'
UNION ALL
SELECT 3, 'Robert', 'Maths,English', 'CA,IN'
GO
SELECT *
FROM #StudentClasses
GO
SELECT ID, Student, value ,value
FROM #StudentClasses
CROSS APPLY STRING_SPLIT(Classes, ',')
CROSS APPLY STRING_SPLIT(CCode, ',')
This must be put in the very first place: Do not store delimited data! If there is any chance to change your table's design, you should use related side-tables to store data this kind...
Your question is not much better than the one before. Without your expected result any suggestion must be guessing.
What I guess: You want to transform 'Maths,Science,English', 'US,UK,AUS' in a way, that Maths goes along with US, Science along with UK and English matches AUS. Try this
SELECT sc.ID
,sc.Student
,A.[key] AS Position
,A.[value] AS Class
,B.[value] AS CCode
FROM #StudentClasses sc
CROSS APPLY OPENJSON('["' + REPLACE(Classes,',','","') + '"]') A
CROSS APPLY OPENJSON('["' + REPLACE(CCode,',','","') + '"]') B
WHERE A.[key]=B.[key];
You did not tell us your SQL Server's version... But you tagged with Azure. Therefore I assume, that v2016 is okay for you. With a lower version (or a lower compatibility level of the given database) there is no JSON support.
Why JSON at all? This is the best way at the moment to split CSV data and get the fragments together with their position within the array. Regrettfully STRING_SPLIT() does not guarantee to return the expected order. With versions lower than v2016 there are several more or less ugly tricks...
If you need your result side-by-side you should read about conditional aggregation.
use select all or use alias
CREATE TABLE #StudentClasses
(ID INT, Student VARCHAR(100), Classes VARCHAR(100),CCode varchar(30))
INSERT INTO #StudentClasses
SELECT 1, 'Mark', 'Maths,Science,English', 'US,UK,AUS'
UNION ALL
SELECT 2, 'John', 'Science,English', 'BE,DE'
UNION ALL
SELECT 3, 'Robert', 'Maths,English', 'CA,IN'
SELECT *,v1.value as clases,v2.value as codes
FROM #StudentClasses
CROSS APPLY STRING_SPLIT(Classes, ',') v2
CROSS APPLY STRING_SPLIT(CCode,
',') v1
Im using two query's, 1st separated one column to two columns and inserted one table and second query (PIVOT) fetching based on inserted table,
1st Query,
SELECT A.MDDID, A.DeviceNumber,
Split.a.value('.', 'VARCHAR(100)') AS MetReading
FROM (
SELECT MDDID,DeviceNumber,
CAST ('<M>' + REPLACE(Httpstring, ':', '</M><M>') + '</M>' AS XML) AS MetReading
FROM [IOTDBV1].[dbo].[MDASDatas] E
Where E.MDDID = 49101
) AS A CROSS APPLY MetReading.nodes ('/M') AS Split(a);
2nd Query
SELECT * FROM
(
Select ID,MDDID,DeviceNumber,ReceivedDate
, ROW_NUMBER() OVER(PARTITION BY ID ORDER BY (SELECT 1)) AS ID2
, SPLT.MR.value('.','VARCHAR(MAX)') AS LIST FROM (
Select ID,MDDID,DeviceNumber,ReceivedDate
, CAST( '<M>'+REPLACE(MeterReading,',','</M><M>')+'</M>' AS XML) AS XML_MR
From [dbo].[PARSEMDASDatas] E
Where E.MeterReading is Not Null
)E
CROSS APPLY E.XML_MR.nodes('/M') AS SPLT(MR)
)A
PIVOT
(
MAX(LIST) FOR ID2 IN ([1],[2],[3],[4],[5],[6],[7],[8])
)PV
I want 2nd query run based on first query no need to require table.
any help would be appreciated.
Your question is not very clear... And it is a very good example, why you always should add a MCVE, including DDL, sample data, own attempts, wrong output and expected output. This time I do this for you, please try to prepare such a MCVE the next time yourself...
If I get this correctly, your source table includes a CSV column with up to 8 (max?) values. This might be solved much easier, no need to break this up in two queries, no need for an intermediate table and not even for PIVOT.
--create a mockup-table to simulate your situation (slightly shortened for brevity)
DECLARE #YourTable TABLE(ID INT,MDDID INT, DeviceNumber VARCHAR(100),MetReading VARCHAR(2000));
INSERT INTO #YourTable VALUES
(2,49101,'NKLDEVELOPMENT02','DCPL,981115,247484,9409') --the character code and some numbers
,(3,49101,'NKLDEVELOPMENT02','SPPL,,,,,,,,') --eigth empty commas
,(4,49101,'NKLDEVELOPMENT02','BLAH,,,999,,'); --A value somewhere in the middle
--The cte will return the table as is. The only difference is a cast to XML (as you did it too)
WITH Splitted AS
(
SELECT ID
,MDDID
,DeviceNumber
,CAST('<x>' + REPLACE(MetReading,',','</x><x>') + '</x>' AS XML) AS Casted
FROM #YourTable t
)
SELECT s.ID
,s.MDDID
,s.DeviceNumber
,s.Casted.value('/x[1]','varchar(100)') AS [1]
,s.Casted.value('/x[2]','varchar(100)') AS [2]
,s.Casted.value('/x[3]','varchar(100)') AS [3]
,s.Casted.value('/x[4]','varchar(100)') AS [4]
,s.Casted.value('/x[5]','varchar(100)') AS [5]
,s.Casted.value('/x[6]','varchar(100)') AS [6]
,s.Casted.value('/x[7]','varchar(100)') AS [7]
,s.Casted.value('/x[8]','varchar(100)') AS [8]
FROM Splitted s;
the result
ID MDDID DeviceNumber 1 2 3 4 5 6 7 8
2 49101 NKLDEVELOPMENT02 DCPL 981115 247484 9409 NULL NULL NULL NULL
3 49101 NKLDEVELOPMENT02 SPPL
4 49101 NKLDEVELOPMENT02 BLAH 999 NULL NULL
The idea in short:
Each CSV is tranformed to a XML similar to this:
<x>DCPL</x>
<x>981115</x>
<x>247484</x>
<x>9409</x>
Using a position predicate in the XPath, we can call the first, the second, the third <x> easily.
CTE: WITH common_table_expression is answer
you can prepare some data in first query and user in second
WITH cte_table AS
(
SELECT *
FROM sys.objects
)
SELECT *
FROM cte_table
where name like 'PK%'
I have a database with a table that very simplified looks like this
Column 1 Column 2 (varchar)
Vector 1 23^34^45^65
Vector 2 0^54^10^31
Now I want to sum the numbers in the cells of column 2 together. That is, I want it to look like this:
Column 1 Column 2 (varchar)
Vector 1 167
Vector 2 95
How do I do this in SQL?
Perhaps you want something like this:
SELECT a.col1, sum_of_values = SUM(d.val)
FROM (VALUES ('Vector 1', '23^34^45^65'), ('Vector 2', '0^54^10^31')) a (col1, col2)
CROSS APPLY (SELECT CONVERT(xml, '<a>' + REPLACE(a.col2, '^', '</a><a>')+'</a>')) AS b(doc)
CROSS APPLY b.doc.nodes('a') c (item)
CROSS APPLY (SELECT c.item.value('.', 'int')) d (val)
GROUP BY a.col1
Output:
col1 sum_of_values
-------- -------------
Vector 1 167
Vector 2 95
Explanation:
The VALUES clause is a placeholder for your data.
By REPLACE'ing caret (^) with XML tags we can use methods on the xml datatype to efficiently split values.
CROSS APPLY with the nodes() method of the xml datatype returns a new row per item, and an xml column containing the item value.
value('.', 'int') converts the inner text of an <a> element to an int.
The GROUP BY and SUM aggregate these results, and reduce the number of rows back to the original two.
Edit: You could move the split into its own function. Like this:
CREATE FUNCTION dbo.SplitIntOnCaret (#list varchar(max)) RETURNS TABLE AS RETURN
SELECT Position = ROW_NUMBER() OVER (ORDER BY (SELECT NULL))
, Value = item.value('.', 'int')
FROM (SELECT CONVERT(xml, '<a>' + REPLACE(#list, '^', '</a><a>')+'</a>')) AS a(doc)
CROSS APPLY doc.nodes('a') c (item)
So your final query could look like this:
SELECT l.list_id, sum_of_values = SUM(s.value)
FROM dbo.MyLists l
CROSS APPLY dbo.SplitIntOnCaret(l.list) AS s
GROUP BY l.list_id
If you cannot change the way your data is stored, you will have to
Split the String (see Turning a Comma Separated string into individual rows)
Compute the SUM using GROUP BY.