T-SQL string conversion - sql

I have an SNMP message column (formatted as VARCHAR(MAX)) in a SQL table like the one below. Is there a way to convert each message OID into a column/value format?
Message column content sample:
community=PUBLIC, enterprise=1.1.1.1.1.1.1.1.1.1.1, uptime=42170345, agent_ip=1.1.1.1, version=Ver2, ...
Desired result:
community enterprise uptime agent_ip
--------- ---------- ------ --------
PUBLIC 1.1.1.1.1.1.1.1.1.1.1 42170345 1.1.1.1 ...
So basically it would need to split the string by ", " and then return INI values as columns. Note this is on one row (not creating or splitting to multiple rows, just multiple columns)
This is SQL Server 2008 R2.
Thank you.

You can find a splitstring function on the web in many places. Here is how you would use it in a query to do what you want:
select t.*, cols.*
from table t cross apply
(select max(case when token like 'community=%' then substring(token, 11, len(token))
end) as community,
max(case when token like 'enterprise=%' then substring(token, 12, len(token))
end) as enterprise,
max(case when token like 'uptime=%' then substring(token, 8, len(token))
end) as uptime,
max(case when token like 'agent_ip=%' then substring(token, 10, len(token))
end) as agent_ip
from dbo.SplitString(t.snmp, ',')(idx, token)
) cols;

Probably not the most efficient way to do this, but this works:
SELECT
REPLACE((SUBSTRING(MsgText,CHARINDEX('community=',MsgText),CHARINDEX(', enterprise=',MsgText) - CHARINDEX('community=',MsgText))),'community=','') AS community
,REPLACE((SUBSTRING(MsgText,CHARINDEX('enterprise=',MsgText),CHARINDEX(', uptime=',MsgText) - CHARINDEX('enterprise=',MsgText))),'enterprise=','') AS enterprise
,REPLACE((SUBSTRING(MsgText,CHARINDEX('uptime=',MsgText),CHARINDEX(', agent_ip=',MsgText) - CHARINDEX('uptime=',MsgText))),'uptime=','') AS uptime
,REPLACE((SUBSTRING(MsgText,CHARINDEX('agent_ip=',MsgText),CHARINDEX(', version=',MsgText) - CHARINDEX('agent_ip=',MsgText))),'agent_ip=','') AS agent_ip
,MsgText
FROM Database.dbo.Table
In case anyone needs a method to parse SNMP messages

Here is solution using transforming string to XML which brings more freedom with result processing:
-- Prepare data for solution testing
DECLARE #srctable TABLE (
Id INT,
SnmpMessage VARCHAR(MAX),
SnmpMessageXml XML
)
INSERT INTO #srctable
SELECT Id, SnmpMessage, SnmpMessageXml FROM ( VALUES
(1, 'community=PUBLIC, enterprise=1.1.1.1.1.1.1.1.1.1.1, uptime=42170345, agent_ip=1.1.1.1, version=Ver2', null)
) v (Id, SnmpMessage, SnmpMessageXml)
-- Transform source formatted string to XML string
UPDATE #srctable
SET SnmpMessageXml = CAST('<row><data ' + REPLACE(REPLACE(SnmpMessage, ',', '"/><data '), '=', '="') + '"/></row>' AS XML)
-- Final select from XML data
SELECT SnmpMessageXml.value('(/row/data/#community)[1]', 'VARCHAR(999)') AS community,
SnmpMessageXml.value('(/row/data/#enterprise)[1]', 'VARCHAR(999)') AS enterprise,
SnmpMessageXml.value('(/row/data/#uptime)[1]', 'VARCHAR(999)') AS uptime,
SnmpMessageXml.value('(/row/data/#agent_ip)[1]', 'VARCHAR(999)') AS agent_ip,
SnmpMessageXml.value('(/row/data/#version)[1]', 'VARCHAR(999)') AS version
FROM #srctable AS t

Related

Oracle Parse NCLOB data to Output or New Table

I have an Oracle 11.2.0.4.0 table named LOOKUPTABLE with 3 fields
LOOKUPTABLEID NUMBER(12)
LOOKUPTABLENM NVARCHAR2(255)
LOOKUPTABLECONTENT NCLOB
The data in the NCLOB field is highly validated on insert so I'm certain the data always is a comma separated string with a CRLF on the end so reads exactly like a simple CSV file. Example ([CRLF] is representation of an actual CRLF, not text)
WITH lookuptable AS (
SELECT
1 AS "LOOKUPTABLEID",
'CODES.TBL' AS "LOOKUPTABLENM",
TO_NCLOB('851,ALL HOURS WORKED GLASS,G,0,,,,,,'||chr(10)||chr(13)||
'935,ALL OT AND HW HRS,G,0,,,,,,'||chr(10)||chr(13)||
'934,ALL PAID TIME,G,0,,,,,,'||chr(10)||chr(13)) AS "LOOKUPTABLECONTENT"
FROM dual
)
SELECT lookuptablecontent FROM lookuptable WHERE lookuptablenm='CODES.TBL';
"851,ALL HOURS WORKED GLASS,G,0,,,,,,[CRLF]935,ALL OT AND HW HRS,G,0,,,,,,[CRLF]934,ALL PAID TIME,G,0,,,,,,[CRLF]"
I essentially want to have a query that can output 1 row for each line in the CLOB. I'm using an application that will read this SQL and write it to a text file for me but it cannot handle CLOB data types and I don't have the option to write directly to file from SQL itself. I have to have a query that can produce this result and allow my app to write the file. I do have the ability to create/write my own tables so a procedure that would read the CLOB into a new table and then I would select from that table in my application would be acceptable if that's better, its just over my head right now. Desired output below, thanks in advance for any help :)
1. 851,ALL HOURS WORKED GLASS,G,0,,,,,,
2. 935,ALL OT AND HW HRS,G,0,,,,,,
3. 934,ALL PAID TIME,G,0,,,,,,
This is a specific case of a general question "how to split a string", and I link this question a lot for more details on that. In this case, instead of a comma, the delimiter that you want to split on is CRLF, or chr(10)||chr(13).
Here's a simple solution with regexp_substr. It's not the fastest solution, but it works fine in simple scenarios. If you need better performance, see the version in the link above with a recursive CTE and no regexp.
WITH lookuptable AS (
SELECT
1 AS LOOKUPTABLEID,
'CODES.TBL' AS LOOKUPTABLENM,
TO_NCLOB('851,ALL HOURS WORKED GLASS,G,0,,,,,,'||chr(10)||chr(13)||
'935,ALL OT AND HW HRS,G,0,,,,,,'||chr(10)||chr(13)||
'934,ALL PAID TIME,G,0,,,,,,'||chr(10)||chr(13)) AS LOOKUPTABLECONTENT
FROM dual
)
SELECT lookuptableid as id, to_char(regexp_substr(lookuptablecontent,'[^('||chr(13)||chr(10)||')]+', 1, level))
FROM lookuptable
WHERE lookuptablenm='CODES.TBL'
connect by level <= regexp_count(lookuptablecontent, '[^('||chr(13)||chr(10)||')]+')
and PRIOR lookuptableid = lookuptableid and PRIOR SYS_GUID() is not null -- needed if more than 1 source row
order by lookuptableid, level
;
Output:
id r
1 851,ALL HOURS WORKED GLASS,G,0,,,,,,
1 935,ALL OT AND HW HRS,G,0,,,,,,
1 934,ALL PAID TIME,G,0,,,,,,
My example data and format using the recursive CTE without regexp from link provided by #kfinity
WITH lookuptable (lookuptableid, lookuptablenm, lookuptablecontent) AS (
SELECT
1,
'CODES.TBL',
TO_NCLOB('ID,NAME,TYPE,ISMONEYSW,EARNTYPE,EARNCODE,RATESW,NEGATIVESW,OVERRIDEID,DAILYSW'||chr(13)||chr(10)||
'851,ALL HOURS WORKED GLASS,G,0,,,,,,'||chr(13)||chr(10)||
'935,ALL OT AND HW HRS,G,0,,,,,,'||chr(13)||chr(10)||
'934,ALL PAID TIME,G,0,,,,,,'
)
FROM dual
), CTE (lookuptableid, lookuptablenm, lookuptablecontent, startposition, endposition) AS (
SELECT
lookuptableid,
lookuptablenm,
lookuptablecontent,
1,
INSTR(lookuptablecontent, chr(13)||chr(10))
FROM lookuptable
WHERE lookuptablenm = 'CODES.TBL'
UNION ALL
SELECT
lookuptableid,
lookuptablenm,
lookuptablecontent,
endposition + 1,
INSTR(lookuptablecontent, chr(13)||chr(10), endposition+1)
FROM CTE
WHERE endposition > 0
)
SELECT
lookuptableid,
lookuptablenm,
SUBSTR(lookuptablecontent, startposition, DECODE(endposition, 0, LENGTH(lookuptablecontent) + 1, endposition) - startposition) AS lookuptablecontent
FROM CTE
ORDER BY lookuptableid, startposition;

Azure Stream analytics default field values for missing fields

I have some json values coming in from an IOT datasource to stream analytics. They want to change the json in a later version to have extra fields but older versions will not have these fields. Is there a way I can detect the field is missing and set up a default value for it before it gets to the output? for example they would like to add an e.OSversion which if it did not exist would default to "unknown". The output is a sql database as it happens.
WITH MetricsData AS
(
SELECT * FROM [MetricsData]
PARTITION BY LID
WHERE RecordType='UseList'
)
SELECT
e.LID as LID
,e.EventEnqueuedUtcTime AS SubmitDate
,CAST (e.UsedDate as DateTime) AS UsedDate
,e.Version as Version
,caUsedList.ArrayValue.Module AS Module
,caUsedList.ArrayValue.UsageCount AS UsedCount
INTO
[ModuleUseOutput]
FROM
Usagedata as e
CROSS APPLY getElements (e.UsedList) as caUsedList
Please use case..when.. operator.
Example:
select j.id, case when j.version is null then 'unknown' else j.version end as version
from jsoninput as j
Output:
Or you could just set the default value in the sql database column directly.

Update UDF names stored in table to add parameter value

I have thousands of UDF names stored in table and executed dynamically where it is required. The problem is I have added one new parameter unit to the function dbo.GetStockPrice(6544,1) so I need to send one more parameter value for now 1 bue it can be any and the data should be changed to dbo.GetStockPrice(6544,1,1) for all the rows where dbo.GetStockPrice is exist. So I am seeking for the query to update these all at once.
Sample Data
DECLARE #table AS TABLE(id INT, UDF VARCHAR(1000))
INSERT INTO #table VALUES
(7774,'dbo.GetStockPrice(1211,1)*dbo.GetStockPrice(1211,1)'),
(7775,'dbo.GetStockPrice(232,1)'),
(7778,'dbo.GetStockPrice(6456,1)'),
(7780,'dbo.GetStockPrice(34,1)'),
(7784,'dbo.FNACondition(dbo.FNAMargin(1,NULL,0), 0, dbo.GetStockPrice(654,1)+1)'),
(7786,'dbo.GetStockPrice(9876,1)'),
(7906,'dbo.GetStockPrice(5565,1)'),
(7911,'dbo.GetStockPrice(7886,1)'),
(7912,'dbo.GetStockPrice(87,1)'),
(8403,'dbo.PriceValue(479,NULL,NULL)*dbo.GetStockPrice(6544,1)+dbo.FNAMargin(1,NULL,0)')
Expected Output:
7774 dbo.GetStockPrice(1211,1,1)*dbo.GetStockPrice(1211,1,1)
7775 dbo.GetStockPrice(232,1,1)
so on......
I am still trying with REPLACE, SUBSTRING but unable to come out with any solution. Getting difficulties with it's different length and position in the row.
Seeking Help !! Thank you in Advance :)
Try it with this approach:
DECLARE #table AS TABLE(id INT, UDF VARCHAR(1000))
INSERT INTO #table VALUES
(7774,'dbo.GetStockPrice(1211,1)*dbo.GetStockPrice(1211,1)'),
(7775,'dbo.GetStockPrice(232,1)'),
(7778,'dbo.GetStockPrice(6456,1)'),
(7780,'dbo.GetStockPrice(34,1)'),
(7784,'dbo.FNACondition(dbo.FNAMargin(1,NULL,0), 0, dbo.GetStockPrice(654,1)+1)'),
(7786,'dbo.GetStockPrice(9876,1)'),
(7906,'dbo.GetStockPrice(5565,1)'),
(7911,'dbo.GetStockPrice(7886,1)'),
(7912,'dbo.GetStockPrice(87,1)'),
(7913,'dbo.Blah(87,1)'),
(8403,'dbo.PriceValue(479,NULL,NULL)*dbo.GetStockPrice(6544,1)+dbo.FNAMargin(1,NULL,0)');
--The query
WITH SplitToParts AS
(
SELECT t.*
,A.parted
,ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) AS PartNr
,B.part.value('text()[1]','nvarchar(max)') AS Part
FROM #table AS t
CROSS APPLY(SELECT CAST('<x>' + REPLACE((SELECT t.UDF AS [*] FOR XML PATH('')),'dbo.GetStockPrice(','</x><x>$$$FoundIt$$$</x><x>') + '</x>' AS XML)) AS A(parted)
CROSS APPLY parted.nodes(N'/x') AS B(part)
WHERE UDF LIKE '%dbo.GetStockPrice(%'
)
,modified AS
(
SELECT *
,CASE WHEN LAG(stp.Part) OVER(PARTITION BY id ORDER BY PartNr)='$$$FoundIt$$$' THEN STUFF(stp.Part,CHARINDEX(')',stp.Part),0,',1') ELSE stp.Part END AS Added
FROM SplitToParts AS stp
)
SELECT t2.*
,(
SELECT REPLACE(m.added,'$$$FoundIt$$$','dbo.GetStockPrice(')
FROM modified AS m
WHERE m.id=t2.id
ORDER BY m.PartNr
FOR XML PATH(''),TYPE
).value('.','nvarchar(max)')
FROM #table AS t2
WHERE UDF LIKE '%dbo.GetStockPrice(%';
The result
7774 dbo.GetStockPrice(1211,1,1)*dbo.GetStockPrice(1211,1,1)
7775 dbo.GetStockPrice(232,1,1)
7778 dbo.GetStockPrice(6456,1,1)
7780 dbo.GetStockPrice(34,1,1)
7784 dbo.FNACondition(dbo.FNAMargin(1,NULL,0), 0, dbo.GetStockPrice(654,1,1)+1)
7786 dbo.GetStockPrice(9876,1,1)
7906 dbo.GetStockPrice(5565,1,1)
7911 dbo.GetStockPrice(7886,1,1)
7912 dbo.GetStockPrice(87,1,1)
8403 dbo.PriceValue(479,NULL,NULL)*dbo.GetStockPrice(6544,1,1)+dbo.FNAMargin(1,NULL,0)
Some explanation: The string will be cut in parts using your function's name as splitter. Gladfully you tagged this with [sql-server-2012] so you can use LAG(). This will test the previous element, if it is $$$FoundIt$$$. In this case the first closing bracket will get an additional ,1. The rest is reconcatenation.
Attention: If your call might include a computed value such as
dbo.GetStockPrice(1211,(1+2))
or
dbo.GetStockPrice(dbo.SomeOtherFunc(1),1)
...the first closing bracket is the wrong place to insert the ,1. But this would get really tricky... You'd have to run through it, char by char, and count the opening brackets to find the related closing one.

Retrieving values from Json in SQL Server

I have a table , which has a column with json Data
[
{"Name":"Territory","Value":"1000000002"},{"Name":"Village","Value":"Bahaibalpur"},
{"Name":"Activity Date","Value":"2016-6-15 5:30:0"},
{"Name":"Start Time","Value":"16:37"},
{"Name":"End Time","Value":"17:38"},
{"Name":"Duration","Value":"1hrs 1 mins"}
]
. I want to get Start time value and end time value inside function.
Try this way
declare #t table(col varchar(1000))
insert into #t(col)
select '[{"Name":"Territory","Value":"1000000002"},{"Name":"Village","Value":"Bahaibalpur"},
{"Name":"Activity Date","Value":"2016-6-15 5:30:0"},
{"Name":"Start Time","Value":"16:37"},
{"Name":"End Time","Value":"17:38"},
{"Name":"Duration","Value":"1hrs 1 mins"}]'
select
substring(col,charindex('Start Time',col)+21,5) as start_time,
substring(col,charindex('end Time',col)+19,5) as end_time
from #t
You don't state which version of SQL you are using, 2016 has built in functionality(https://msdn.microsoft.com/en-us/library/dn921897.aspx)
Otherwise, there is a cool article on parsing Json with TSQL that you can read here.
The other option is to use a CLR function.

Querying XML colum for values

I have a SQL Server table with an XML column, and it contains data something like this:
<Query>
<QueryGroup>
<QueryRule>
<Attribute>Integration</Attribute>
<RuleOperator>8</RuleOperator>
<Value />
<Grouping>OrOperator</Grouping>
</QueryRule>
<QueryRule>
<Attribute>Integration</Attribute>
<RuleOperator>5</RuleOperator>
<Value>None</Value>
<Grouping>AndOperator</Grouping>
</QueryRule>
</QueryGroup>
</Query>
Each QueryRule will only have one Attribute, but each QueryGroup can have many QueryRules. Each Query can also have many QueryGroups.
I need to be able to pull all records that have one or more QueryRule with a certain attribute and value.
SELECT *
FROM QueryBuilderQueries
WHERE [the xml contains any value=X where the attribute is either Y or Z]
I've worked out how to check a specific QueryRule, but not "any".
SELECT
Query
FROM
QueryBuilderQueries
WHERE
Query.value('(/Query/QueryGroup/QueryRule/Value)[1]', 'varchar(max)') like 'UserToFind'
AND Query.value('(/Query/QueryGroup/QueryRule/Attribute)[1]', 'varchar(max)') in ('FirstName', 'LastName')
You can use two exist(). One to check the value and one to check Attribute.
select Q.Query
from dbo.QueryBuilderQueries as Q
where Q.Query.exist('/Query/QueryGroup/QueryRule/Value/text()[. = "UserToFind"]') = 1 and
Q.Query.exist('/Query/QueryGroup/QueryRule/Attribute/text()[. = ("FirstName", "LastName")]') = 1
If you really want the like equivalence when you search for a Value you can use contains().
select Q.Query
from dbo.QueryBuilderQueries as Q
where Q.Query.exist('/Query/QueryGroup/QueryRule/Value/text()[contains(., "UserToFind")]') = 1 and
Q.Query.exist('/Query/QueryGroup/QueryRule/Attribute/text()[. = ("FirstName", "LastName")]') = 1
According to http://technet.microsoft.com/pl-pl/library/ms178030%28v=sql.110%29.aspx
"The XQuery must return at most one value"
If you are quite certain that for example your XML has let's say maximum 10 QueryRules you could maybe use WHILE to loop everything while droping your results into temporary table?
maybe below can help you anyway
CREATE TABLE #temp(
Query type)
DECLARE #i INT
SET #i = 1
WHILE #i >= 10
BEGIN
INSERT INTO #temp
SELECT
Query
FROM QueryBuilderQueries
WHERE Query.value('(/Query/QueryGroup/QueryRule/Value)[#i]', 'varchar(max)') LIKE 'UserToFind'
AND Query.value('(/Query/QueryGroup/QueryRule/Attribute)[#i]', 'varchar(max)') IN ('FirstName', 'LastName')
#i = #i + 1
END
SELECT
*
FROM #temp
It's a pity that the SQL Server (I'm using 2008) does not support some XQuery functions related to string such as fn:matches, ... If it supported such functions, we could query right inside XQuery expression to determine if there is any. However we still have another approach. That is by turning all the possible values into the corresponding SQL row to use the WHERE and LIKE features of SQL for searching/filtering. After some experiementing with the nodes() method (used on an XML data), I think it's the best choice to go:
select *
from QueryBuilderQueries
where exists( select *
from Query.nodes('//QueryRule') as v(x)
where LOWER(v.x.value('(Attribute)[1]','varchar(max)'))
in ('firstname','lastname')
and v.x.value('(Value)[1]','varchar(max)') like 'UserToFind')