I have a database column containing a string that might look something like this u/1u/3u/19/g1/g4 for a particular row.
Is there a performant way to get all rows that have at least one of the following elements ['u/3', 'g4'] in that column?
I know I can use AND clauses, but the number of elements to verify against varies and could become large..
I am using RoR/ActiveRecord in my project.
in sql server, you can use XML to convert your list of search params into a record set, then cross join that with the base table, and do charIndex() to see if the column contains the substring.
Since i don't know your table or column names, i used a table (persons) that i already had data in, which has a column 'phone_home'. To search for any phone number that contains '202' or '785' i would use this query:
select person_id,phone_home,Split.data.value('.', 'VARCHAR(10)')
from (select *, cast('<n>202</n><n>785</n>' as XML) as myXML
from persons) as data cross apply myXML.nodes('/n') as Split(data)
where charindex(Split.data.value('.', 'VARCHAR(10)'),data.phone_Home) > 0
you will get duplicate records if it matches more than one value, so throw a distinct in there and remove the Split from the select statement if that is not desired.
Using xml in sql is voodoo magic to me...i got the idea from this post http://www.sqljason.com/2010/05/converting-single-comma-separated-row.html
no idea what performance is like...but at least there aren't any cursors or dynamic sql.
EDIT: Casting the XML is pretty slow, so i made it a variable so it only gets cast once.
declare #xml XML
set #xml = cast('<n>202</n><n>785</n>' as XML)
select person_id,phone_home,Split.persons.value('.', 'VARCHAR(10)')
from persons cross apply #xml.nodes('/n') as Split(persons)
where charindex(Split.persons.value('.', 'VARCHAR(10)'),phone_Home) > 0
Related
How do I remove duplicates in the following case in T-SQL?
I have a table with a column Code of type varchar(max).
It contains column value like truck/rail/truck/rail. I need the cell value to be truck/rail.
Other possibility is truck/rail/ship/truck need to be truck/rail/ship.
By using table valued function.
Thanks.
You can use String_Split along with String_agg to remove the duplicates.
DECLARE #t table(id int, val varchar(max))
insert into #t values(1,'truck/rail/truck/rail'), (2,'truck/rail/ship/truck')
SELECT t.id,STRING_AGG(splitval,'/') as newval FROM #t as t
cross apply (
SELECT distinct value from string_split(t.val,'/')) as ca(splitval)
group by t.id
id
newval
1
rail/truck
2
rail/ship/truck
Note1: String_Split, does not guarantee order. So, your concatenated results might be in different order from the original list, after duplicates removal. If you want to preserve the order, then we have to go for different solution using xml nodes or json array.
Note2: String_Split was introduced in SQL Server 2016. String_agg was introduced in SQL Server 2017. So, if you are using versions before that, you have to go for recursive CTE and CHARINDEX based solution.
If you know that the error exists, then just do an UPDATE where you replace the truck/rail/truck/rail with truck/rail using the REPLACE(Code,'truck/rail/truck/rail',truck/rail).
The same goes for your truck/rail/ship/truck issue.
If you need automatic detection and correction to be done, that's a whole 'nuther story but could still be done using nested REPLACES. Detection of the issue is the hard part. Personally, I'd be having a talk with the people that are providing the data.
I have a column in a table that is json. It contains several columns within it.
Example:
Row1: "sTCounts":[{"dpsTypeTest":"TESTTRIAL","cnt":3033244.0}
Row2: "sTCounts":[{"dpsTypeTest":"TESTTRIAL","cnt":3.3}
I need to sum the cnt value for all rows in table. For instance, the above would produce a result of 3033247.3
I'm not familiar with stored procs enough to master. I thought the easiest route would be to create a temp table and extract the value into a column, and then write a query to sum the column values.
The problem is that it creates a column with datatype nvarchar(4000). It won't let me sum that column. I thought of changing the datatype but not sure how. I am trying CAST without luck.
select CAST(json AS varchar) AS JSON_VALUE(jsontext,
'$.sTCounts.cnt') AS PerfCount, TitleNumber
INTO dbo_Testing_Count0
from PerformanceTest
select sum(PerfCount)
from dbo_Testing_Count
Group by PerfCount
The error message is:
Incorrect syntax near 'jsontext'.
Any ideas? I am open to another method to sum the column or changing the datatype whichever the experts can aid on. I appreciate it.
The JSON you provide in your question is not valid... This seems to be just a fragment of a larger JSON. As your data starts with a [ you have to think of it as an array, so the simple json path '$.serviceTierCounts.cnt' won't work probably...
Try this, I've added the opening { and the closing brackets at the end:
DECLARE #mockupTable TABLE(ID INT IDENTITY, YourJson NVARCHAR(MAX));
INSERT INTO #mockupTable VALUES
(N'{"serviceTierCounts":[{"dpsType":"TRIAL","cnt":3033244.0}]}')
,(N'{"serviceTierCounts":[{"dpsType":"TRIAL","cnt":3.3}]}');
--You can read one scalar value using JSON_VALUE directly with a cast. But in this case I need to add [0]. This will tell the engine to read the first (zero-based index!) object's cnt property.
SELECT CAST(JSON_VALUE(YourJson,'$.serviceTierCounts[0].cnt') AS DECIMAL(14,4))
FROM #mockupTable
--But I think, that it's this what you are looking for:
SELECT *
FROM #mockupTable
CROSS APPLY OPENJSON(YourJson,'$.serviceTierCounts')
WITH(dpsType varchar(100)
,cnt decimal(14,4));
The WITH clause will return the object in typed columns side-by-side.
For easy proceeding, you can wrap this as a CTE and continue with the set in the following SELECT.
I have a scenario where a table, containing an XML column, will occasionally contain XML data which displays the following error when trying to select it:
"Msg 6611, Level 16, State 1, Line 889
The XML data type is damaged."
I have pin pointed the record with the issue, but it's not something I want to correct at this time but I instead want my select statement to still select data from the table without the whole thing falling over. If there was a way to use a function to check whether the XML is damaged/ valid, if so select a blank value, then this would be my preferred option or if not perhaps just exclude the troublesome record from the results.
Hopefully this made sense and thank you in advance!
Example query which errors (If I exclude the XML column, the query works):
SELECT
T.ID,
T.XML
FROM
TableName T
Ultimatley, this is the kind of query I'm trying to run (I'd like the value field to display as blank if the XML is damaged):
SELECT
T.ID,
t.c.value ('(Value)[1]', 'varchar(50)') AS Value
FROM
TableName T
CROSS APPLY DocumentXML.nodes('//XMLNode') t(c)
I have a column of type 'nvarchar(max)' that should now hold XML information instead of just a string.
Say: col1 has value 'abc'
Now it has values, with additional info:
<el1>abc</el2>
<el2>someotherinfo</el2>
Storing the information to the column is fine, since it can still be pushed in as a string.
However, extracting the same information and also using/replacing the same information 'abc' from this column that is being used in various other joins from other tables, is something I'm not able to figure out.
how can I also push in this information into abcd when it comes from another table's value 'abcd' without losing other information?
I am building an XML from the application side and updating it in a column of type nvarchar(). All the columns have been replaced to hold the XML, so the safe assumption is that the col1 only holds XML similar to that mentioned above. Just push the XML as is and it works fine. However, how should I extract the information to use it in joins?
How do I extract a particular element from this nvarchar() string to use it in a join??
Previously, this column 'Col1' was just used as a string, and a check was done like this:
where tablex.colx = table1.col1
or
Update Table2 where
Once you cast the NVARCHAR data to the XML data type, you can use XML functions to get element/attribute values for joining to:
WITH xoutput AS (
SELECT CONVERT(xml, t.nvarchar_column) AS col
FROM YOUR_TABLE t)
SELECT x.*
FROM TABLE x
JOIN xoutput y ON y.col.value('(/path/to/your/element)[1]', 'int') = x.id
It won't be able to use indexes, because of the data type conversion...
Alternate version, using IN:
WITH xoutput AS (
SELECT CONVERT(xml, t.nvarchar_column) AS col
FROM YOUR_TABLE t)
SELECT x.*
FROM TABLE x
WHERE x.id IN (SELECT y.col.value('(/path/to/your/element)[1]', 'int')
FROM xoutput)
If you have a varchar field you can easily do SELECT * FROM TABLE WHERE ColumnA LIKE '%Test%' to see if that column contains a certain string.
How do you do that for XML Type?
I have the following which returns only rows that have a 'Text' node but I need to search within that node
select * from WebPageContent where data.exist('/PageContent/Text') = 1
Yet another option is to cast the XML as nvarchar, and then search for the given string as if the XML vas a nvarchar field.
SELECT *
FROM Table
WHERE CAST(Column as nvarchar(max)) LIKE '%TEST%'
I love this solution as it is clean, easy to remember, hard to mess up, and can be used as a part of a where clause.
This might not be the best performing solution, so think twice before deplying it to production. It is however very usefull for a quick debug session, which is where I mostly use it.
EDIT: As Cliff mentions it, you could use:
...nvarchar if there's characters that don't convert to varchar
You should be able to do this quite easily:
SELECT *
FROM WebPageContent
WHERE data.value('(/PageContent/Text)[1]', 'varchar(100)') LIKE 'XYZ%'
The .value method gives you the actual value, and you can define that to be returned as a VARCHAR(), which you can then check with a LIKE statement.
Mind you, this isn't going to be awfully fast. So if you have certain fields in your XML that you need to inspect a lot, you could:
create a stored function which gets the XML and returns the value you're looking for as a VARCHAR()
define a new computed field on your table which calls this function, and make it a PERSISTED column
With this, you'd basically "extract" a certain portion of the XML into a computed field, make it persisted, and then you can search very efficiently on it (heck: you can even INDEX that field!).
Marc
Another option is to search the XML as a string by converting it to a string and then using LIKE. However as a computed column can't be part of a WHERE clause you need to wrap it in another SELECT like this:
SELECT * FROM
(SELECT *, CONVERT(varchar(MAX), [COLUMNA]) as [XMLDataString] FROM TABLE) x
WHERE [XMLDataString] like '%Test%'
This is what I am going to use based on marc_s answer:
SELECT
SUBSTRING(DATA.value('(/PAGECONTENT/TEXT)[1]', 'VARCHAR(100)'),PATINDEX('%NORTH%',DATA.value('(/PAGECONTENT/TEXT)[1]', 'VARCHAR(100)')) - 20,999)
FROM WEBPAGECONTENT
WHERE COALESCE(PATINDEX('%NORTH%',DATA.value('(/PAGECONTENT/TEXT)[1]', 'VARCHAR(100)')),0) > 0
Return a substring on the search where the search criteria exists