Handling Json data in snowflake - sql

enter image description here
I have a table which contains Json file data in each row which gets updated into my snowflake table every weak. I am extracting values from the Json files into another table. When the data is loaded in Json format there are multiple entries of the same ID. So, when I extract values from Json to a table there are duplicate rows. How do I tackle them in order to get the distinct rows only. My select query look something like this:
select
json_data:data[0].attributes."Additional Invoice?":: string as "Additional Invoice?",
json_data:data[0].attributes."Additional PO?":: string as "Additional PO?",
json_data:data[0].attributes."Aggregate Contract Value":: number as "Aggreagate Contract Value" ,
json_data:data[0].attributes."Annualized Baseline Spend" :: number as "Annualized Baseline Spend",
json_data:data[0].id ::number as ID,
json_data:data[0].type::string as TYPE
from scout_projects order by ID
the scout project file screenshot is attached.
The attached Screenshot is the output form the given query and as you could see the ID column is the same but there are only 2 unique rows. I want my query to return only those 2 unique rows.
select distinct json_data:data[0].id :: number as ID from scout_projects
what is the approach should I take?
I tried using subquery, but it gave me error stating "single-row subquery returns more than one row. snowflake error" which is obvious. so, need a way out .

Related

Pivoting previously xml data via SQL query throws error

I have data in table as below where value column data is quite big, like unstructured text:
http://s3.pdfconvertonline.com/convert/p3r68-cdx67/78gbs-hvj2r.html
The characters which you find in date like &amp and &nbsp are present and this is just for 2 small records, actual data is quite bigger which is why i use pivot xml as the IDs are 300 in real data set.
The Heading and Value columns were initially HTML data for each ID which is now split on basis of heading and corresponding value in html using xmltype parsing.
Now we have data in the 2 columns split.
I need to pivot this, i.e. the Heading column values which are constant for ever id to become column headers and the respective values to come below as rows.
When I run the pivot query it throws error:
select *
from data
pivot xml (max(id) for heading in (select heading from data));
An error occurs in XML parsing.
Entity reference is not well formed.
XML Parser returned an error while trying to parse the document.
Check if document to be parsed is valid.
Could the error be because of these special characters?

Why The Query Against HashKey returns no records

I am working on a new sql table. The table has a column [varbinary(8000)], where we are storing hash of a certain text. Now, I am trying to retrieve the same record back by using a where clause against the hashkey, but that yields zero records.
I have added a similar query here: http://sqlfiddle.com/#!18/be996/11
Try without the single quotes, like this
SELECT id, description
FROM ForgeRock
where id = 0x94EE059335E587E501CC4BF90613E0814F00A7B08BC7C648FD865A2AF6A22CC2
and you will get the expected result.

How do I select a value in a key:value pair within a list in a column using SQL?

In a table called payouts, there is a column stripeResponseData where the data is in the following structure:
{"id":"tr_1BlSHbGQXLV7RqqnHJffUVO0","object":"transfer","amount":39415,"amount_reversed":0,"balance_transaction":"txn_1BlSHbGQXfV7AqqnGi2o7UiY","created":1516239215,"currency":"usd","description":null,"destination":"acct_1BWWAmAzms5xPfV9","destination_payment":"py_1BlSHbAzms5xkfV91RHAOrno","livemode":true,"metadata":{},"reversals":{"object":"list","data":[],"has_more":false,"total_count":0,"url":"/v1/transfers/tr_1BlSHbYQXLV7AqqnHJffUVO0/reversals"},"reversed":false,"source_transaction":null,"source_type":"card","transfer_group":null}
Within my SQL SELECT statement, I want to return only the value of the key "destination". How do I write my SQL query?
My desired result of the query:
SELECT "stripeResponseData" FROM payouts [...]
(where I don't know how to write [...]) should look like the following (assume we have 3 rows with different values on "destination"):
acct_1BWWAmAzms5xPfV9
acct_1AY0phDc9pCDpLR8
acct_1AwG3VL7DXxftOaS
How do I extract that value from the list within the stripeResponseData column?
See this sqlfiddle. This query will fetch the ID from stripResponseData where the id is a specific id (Probably not very useful, but does show you how to select and query):
SELECT data->>'id' FROM stripeResponseData WHERE data #> '{"id":"tr_1BlSHbGQXLV7RqqnHJffUVO0"}';
Because you mentioned your data was a string, you need to to type conversions to query/use it correctly. See this sqlfiddle:
SELECT data::jsonb->>'id' FROM stripeResponseData WHERE data::jsonb #> '{"id":"tr_1BlSHbGQXLV7RqqnHJffUVO0"}';
Per your edit, you can simply query destination in almost the exact same way. This will get all the id's from stripeResponseData where destination = acct_1BWWAmAzms5xPfV9:
SELECT data::jsonb->>'id' FROM stripeResponseData WHERE data::jsonb #> '{"destination":"acct_1BWWAmAzms5xPfV9"}';

Creating a calculated field table based on data in separate tables

It is straight forward to create a calculated field in a table that uses data IN the table... due to the fact that the expression builder is straight forward to use. However, it appears to me that the expression builder for the calculated field only works with data IN the table;
i.e: expression builder in table MYTABLE works with fields FIELD1.MYTABLE, FIELD2.MYTABLE etc.
Inventory Problem
My problem is that I have two 'count' fields that result from my queries that apply to INPUTQUERY and OUTPUTQUERY (gives me a count of all input data added and a count of all output data added) and now I want to subtract the two to get a stock.
I can't link the table that was created from my query because it wont be able to continually update do the relationship itself, and thus i'm stuck either using the expression builder/SQL.
First question:
Is it possible to have the expression builder reference data from other tables?
i.e expressionbuilder for:
MAINTABLE CALCULATEDFIELD.MAINTABLE = INPUTSUM.INPUTTABLE - OUTPUTSUM.OUTPUTTABLE
(which gives a difference of the two)?
Second question:
if the above isn't possible, can I do this through an SQL code ?
i.e
SELECT(data from INPUTSUM)
FROM(INPUTTABLE)
-
SELECT(data from OUTPUTSUM)
FROM(OUTPUTTABLE)
Try this:
SELECT SUM(T.INPUTSUM) - SUM(T.OUTPUTSUM) AS RESULTSUM
FROM
(
SELECT INPUTSUM, 0 AS OUTPUTSUM
FROM INPUTTABLE
UNION
SELECT 0 AS INPUTSUM, OUTPUTSUM
FROM OUTPUTTABLE
) AS T

Compare XML data to String

I have a table that houses a bunch of data in an XML field. I can get to the data and display what I need in the select statement, but I also need to use that to compare to another table that houses a translation I am trying to do. Is there a way to compare the value being returned from the XML data to a string value that exists in another table?
The code in my select to return the XML data is:
prv.reported_attributes.value('(/row[#ATTRIBUTE="FIELD"][1])/#VALUE', 'varchar(5)')
I need to compare that text output to another table, but I keep getting NULL like the values I am trying to compare do not match. I have confirmed they do in fact have matches.