I have some json values coming in from an IOT datasource to stream analytics. They want to change the json in a later version to have extra fields but older versions will not have these fields. Is there a way I can detect the field is missing and set up a default value for it before it gets to the output? for example they would like to add an e.OSversion which if it did not exist would default to "unknown". The output is a sql database as it happens.
WITH MetricsData AS
(
SELECT * FROM [MetricsData]
PARTITION BY LID
WHERE RecordType='UseList'
)
SELECT
e.LID as LID
,e.EventEnqueuedUtcTime AS SubmitDate
,CAST (e.UsedDate as DateTime) AS UsedDate
,e.Version as Version
,caUsedList.ArrayValue.Module AS Module
,caUsedList.ArrayValue.UsageCount AS UsedCount
INTO
[ModuleUseOutput]
FROM
Usagedata as e
CROSS APPLY getElements (e.UsedList) as caUsedList
Please use case..when.. operator.
Example:
select j.id, case when j.version is null then 'unknown' else j.version end as version
from jsoninput as j
Output:
Or you could just set the default value in the sql database column directly.
Related
I'm using Query which joins external data through EXTERNAL_QUERY() LIKE THIS
(this is just example, not actual one)
SELECT
ext.program_id,
SUM(price) AS total_price
FROM a_dataset.purchases pcs
LEFT OUTER JOIN (
SELECT
program_id,
version
FROM EXTERNAL_QUERY(
'CONNECTION_INFO',
'SELECT program_id, version FROM products'
)
) ext ON pcs.program_id = ext.program_id
This query actually worked at my environment.
However, from today, this part ↓
EXTERNAL_QUERY(
'CONNECTION_INFO',
'SELECT program_id, version FROM products'
)
starts to return byte value which looks like encrypted and
turns out to show this message
No matching signature for operator = for argument types: STRING, BYTES. Supported signatures: ANY = ANY at [37:9]
'CONNECTION_INFO' refers Cloud SQL, read replica instance of MySQL.
Do you have any ideas how to fix this, or why these return values started to changed ?
I have created this table in Oracle SQL Developer
CREATE TABLE Test_T (
COL_1 VARCHAR(30),
COL_2 XMLTYPE
);
And have inserted this into it
INSERT INTO Test_T VALUES ('two', ('<?xml version="1.0" encoding="UTF-8"?>
<CATALOG>
<PLANT>
<COMMON>Bloodroot</COMMON>
<BOTANICAL>Sanguinaria canadensis</BOTANICAL>
<ZONE>4</ZONE>
<LIGHT>Mostly Shady</LIGHT>
<PRICE>$2.44</PRICE>
<AVAILABILITY>031599</AVAILABILITY>
</PLANT>
<PLANT>
<COMMON>Columbine</COMMON>
<BOTANICAL>Aquilegia canadensis</BOTANICAL>
<ZONE>3</ZONE>
<LIGHT>Mostly Shady</LIGHT>
<PRICE>$9.37</PRICE>
<AVAILABILITY>030699</AVAILABILITY>
</PLANT></CATALOG>'));
My goal is to return the <COMMON> Name </COMMON> ONLY WHERE the zone is 3 or LESS. So this should return Columbine.
I thought about using XMLExists Im not too familiar with XML so this is what I had so far.
SELECT COL_2 FROM Test_T WHERE XMLExists('//ZONE[COL_2 <= 3]' PASSING BY REF COL_2);
I'm not sure if I am accessing the ZONE right.
Could anyone guide me in the right direction?
Try the below select query :
SELECT COMMON_NAME FROM Test_T WHERE XMLExists( 'CATALOG/PLANT[ZONE<=3]/COMMON[text()]' PASSING COMMON_NAME )
The problem is with your path( '//ZONE[COL_2 <= 3]' ). COL_2 is not a valid XML node, it's just the name of your column.
The proper path would be //ZONE[text() <= 3].
text() is a special node reference that tells oracle to grab the text inside the ZONE node <ZONE>THIS TEXT</ZONE>. You can only target nodes in your actual XML schema.
Also, be aware that the path is CASE SENSITIVE to what's in your XML. Remembering that will save you time.
Additionally, another way to write your select would be this. In this example, Oracle does an implicit join and returns a row for each //PLANT with /ZONE/text() <= 3. The path in the XMLSEQUENCE is important here because it determines how oracle splits up each row, meaning that you can't just target //ZONE because you would only get a row for each ZONE rather than a row for each PLANT.
In the select clause, you can extract individual node values for each PLANT if you have more than one.
SELECT VALUE(P) --THE FULL XML FOR EACH PLANT
VALUE(P).EXTRACT('//COMMON/text()').getstringval() AS COMMON, --INDIVIDUAL NODE VALUE
VALUE(P).EXTRACT('//BOTANICAL/text()').getstringval() AS BOTANICAL --INDIVIDUAL NODE VALUE
FROM Test_T, TABLE(XMLSEQUENCE(EXTRACT(COL_2, '//PLANT[ZONE<= 3]'))) p
I have a SQL query that goes like this:
select v FROM v.rsversion between 'minVer' and 'maxVer';
Where the version is expressed as a string of format x.y.z
This will return fine all existing versions between 0.2.0 and 0.2.9
but will return nothing if the range is 0.2.0 and 0.2.10
Is there a way to make this work?
With Postgres you could do this by splitting up the version into three numbers and then compare those numbers. For other DBMS you would need to find a different way of splitting a string like 0.2.1 into three numbers.
with rsversion (v) as (
values
-- only here for sample data
('0.2.0'), ('0.2.1'), ('0.2.2'), ('0.2.10'), ('0.2.12'),
('0.3.0'), ('0.3.1'), ('0.3.2'), ('0.3.4')
), numeric_version (v, major, minor, patch) as (
select v,
split_part(v,'.', 1)::int,
split_part(v,'.', 2)::int, ,
split_part(v,'.', 3)::int
from rsversion
)
select v
FROM numeric_version
where (major,minor,patch) between (0,2,1) and (0,2,11)
The above prints:
v
------
0.2.1
0.2.2
0.2.10
SQLFiddle example: http://sqlfiddle.com/#!15/403d2/2
The reason this is not working as designed is because 0.2.0 and 0.2.x are not literal numbers, so if you are trying to do a string comparison it's looking at each incremented character and comparing them.
So 0.2.0, 0.2.1, 0.2.10, 0.2.2, 0.2.3, etc is how it's arranging the strings.
You may be able to make this work by adding a leading 0 to the third part of the string 0.2.00, 0.2.01, 0.2.02, etc if that is possible to do for your purposes.
Does this work? I'm assuming your version will always be in a similar format and that your minVer and maxVer are stored as text.
WITH
DATA AS
(
SELECT whatever other data you require,
SUBSTRING(v.rsversion FROM '^[0-9]+.[0-9]+'), '.', '')::decimal as version_part_one,
SUBSTRING(v.rsversion FROM '[0-9]+$')::decimal as version_part_two
FROM v
)
SELECT d.whatever other data you require
FROM data d
WHERE d.version_part_one >= SUBSTRING(minVer FROM '^[0-9]+.[0-9]+')::decimal
AND
d.version_part_two >= SUBSTRING(minVer FROM '.[0-9]+$')::decimal
AND
d.version_part_one <= SUBSTRING(minVer FROM '^[0-9]+.[0-9]+')::decimal
AND
d.version_part_two <= SUBSTRING(maxVer FROM '.[0-9]+$')::decimal
You can change the name of the CTE if you're feeling more creative...
All I need to do is simply get one geography value from a table and store it in another table. There is some logic for which row to take from the origin table so it's not just a straight select.
In any of 50 possible variants of this, I get this error when hitting the update to the target table:
Msg 403, Level 16, State 1, Line 1
Invalid operator for data type. Operator equals not equal to, type equals geography.
My SQL looks like this at the moment:
declare
#EquipmentId int
, #CurrentLocationId int
, #CurrentGeoLocation geography
, #LastUpdated datetime
select #EquipmentId =
(
select top 1 EquipmentId
from Equipment
order by EquipmentId
)
select #CurrentLocationId = (select top 1 EquipmentLocationId from EquipmentLocation where EquipmentId = #EquipmentId order by LastUpdated desc)
select #LastUpdated = (select top 1 LastUpdated from EquipmentLocation where EquipmentId = #EquipmentId order by LastUpdated desc)
UPDATE
dbo.Equipment
SET
CurrentLocationDateTime = #LastUpdated
, CurrentGeoLocation = (select GeoLocation from EquipmentLocation where EquipmentLocationId = #CurrentLocationId)
, ModifiedBy = 'system'
, ModifiedByUserId = -1
, ModifiedDate = getdate()
WHERE
EquipmentId = #EquipmentId
I have had CurrentGeoLocation set in a variable of the same type, selected into by the same statement you see in the update.
I have had an #CurrentGeoLocation variable populated by a geography::STGeomFromText as well as geography::Point() function call.
I've used Lat and Long variables to call Point and FromText functions.
All the same result, the above 403 error. I could understand it somewhat when I was concatenating various permutations of the GeomFromText function that needs well known text format for the point parameter, but field value to field value is killing me, as is the fact that I get this error no matter how I try to give the origin point data to the target table.
Thoughts?
Update:
I've been experimenting a little and found that the following works just fine:
declare #GL geography
select #GL = (select GeoLocation from EquipmentLocation where EquipmentLocationId = 25482766)
print convert(varchar, #GL.Lat)
print convert(varchar, #GL.Long)
update Equipment set CurrentGeoLocation = geography::Point(#GL.Lat, #GL.Long, 4326)-- #NewGL where EquipmentId = 10518
But then when I apply this plan to the original script, I'm back to the same error.
The data in the test is working off the exact same records as in the original script. The original script is working off a collection of EquipmentIds, on the first one, I encounter this problem. The short test script uses the same EquipmentLocationId and EquipemntId that are the selected values used to update the first Equipment record in my collection.
Solved!
The error had nothing to do with the geography type as SQL reported. By pulling items in and out of the update statement in an effort to isolate why I still get the error even if I save everything but CurrentGeoLocation and then another update for the geography, I found that CurrentLocationDateTime (datetime, null) was the culprit. Deleted the column, added it back. Problem solved. Original script works as expected.
Don't know what happened to that datetime column that caused it to throw errors against a geometry type, but it's fixed.
Good morning everyone!
Below is a piece of code I stitched together: I used a CTE to grab the records(data) from a link table and than convert strings to dates, than use the merge statement to get the data into a local table:
I am having a problem with the column(field) LAST_RACE_DATE this field is set to NULL and is not required but it does not update with my current set up. What I am trying to accomplished is for this field to populate when data is entered but also update, meaning it should also update with NULL.
So if the field has a specific date, and a new date is entered in the remote database, this field should update as well, even if the data is deleted in the back end, it should also remove the local table data for this field.
WITH CTE AS(
SELECT MEMBER_ID
,[MEMBER_DATE] = MAX(CONVERT(DATE, MEMBER_DATE))
,RACE_DATE = MAX(CONVERT(DATE, RACE_DATE))
,LAST_RACE_DATE = MAX(CONVERT(DATE, LAST_RACE_DATE))
FROM [EXAMPLE].[dbo].[LINKED_MEMBER_DATA]
WHERE (MEMBER_DATE IS NOT NULL) AND (ISDATE(MEMBER_DATE)<> 0) AND (RACE_DATE IS NOT NULL) AND (ISDATE(RACE_DATE)<> 0)
AND (LAST_RACE_DATE IS NULL) OR (ISDATE(LAST_RACE_DATE)<> 0)
GROUP BY MEMBER_ID)
MERGE dbo.LINKED_MEMBER_DATA AS Target
USING (SELECT
MEMBER_ID, MEMBER_DATE, RACE_DATE, LAST_RACE_DATE
FROM CTE
GROUP BY MEMBER_ID, RACE_DATE, LAST_RACE_DATE)AS SOURCE ON (Target.MEMBER_ID = SOURCE.MEMBER_ID)
WHEN MATCHED AND
(Target.MEMBER_DATE) <> (SOURCE.MEMBER_DATE)
OR (Target.RACE_DATE) <> (SOURCE.RACE_DATE)
OR ISNULL(TARGET.LAST_RACE_DATE , Target.LAST_RACE_DATE) <> ISNULL(SOURCE.LAST_RACE_DATE, SOURCE.LAST_RACE_DATE)
THEN UPDATE SET
Target.MEMBER_DATE = SOURCE.MEMBER_DATE
,Target.RACE_DATE = SOURCE.RACE_DATE
,Target.LAST_RACE_DATE = SOURCE.LAST_RACE_DATE
WHEN NOT MATCHED BY TARGET THEN
INSERT(
MEMBER_ID, MEMBER_DATE, RACE_DATE, LAST_RACE_DATE)
VALUES (Source.MEMBER_ID, Source.MEMBER_DATE, Source.RACE_DATE, Source.LAST_RACE_DATE);
I also tried this:
ISNULL(Target.LAST_RACE_DATE,'N/A') <> ISNULL(SOURCE.LAST_RACE_DATE,'N/A')
But it generates the below error for dates conversion:
Conversion failed when converting date and/or time from character string.
Thanks a Million!!
Your current statement is failing because the ISNULLs that you have don't do anything (if one of the values is NULL the expression will evaluate to NULL), and NULL values don't compare. Your second attempt doesn't work because ISNULL requires the data types of the two values to be the same, so you could try eg ISNULL(Target.LAST_RACE_DATE, '1970-01-01') <> ISNULL(Source.LAST_RACE_DATE, '1970-01-01').
Another option would be to simply enumerate the different cases (eg, (((Source.LAST_RACE_DATE IS NULL AND Target.LAST_RACE_DATE IS NOT NULL) OR (Source.LAST_RACE_DATE IS NOT NULL AND Target.LAST_RACE_DATE IS NULL) OR (Source.LAST_RACE_DATE <> Target.LAST_RACE_DATE))). Enumerating the different situations makes the code a bit more verbose, but it can result in better performance (whether it is measurably better really depends on how much data you are processing).