I am querying an Azure SQL table in ADF Lookup activity
My query looks like this
#concat('select distinct ltrim(rtrim(copay)) as column1 from ',item().TABLE_NAME,' where column1 is not null for json auto')
The output in Preview option looks like below
I want to remove highlited random JSON header from output
I am passing this value to Web API CALL and I need value inside Square brackets
activity('GetLookupValues').output.value[0]
One way is to wrap the JSON query in a CAST statement.
#concat( 'SELECT CAST( (SELECT DISTINCT t.CoPay AS column1 FROM ', item().TABLE_NAME, ' t WHERE t.CoPay IS NOT NULL FOR JSON AUTO) AS NVARCHAR(MAX))')
Related
I have table with data like this:
Id | StringValue
----+-------------
1 | 4,50
2 | 90,40
I will get input StringValue like 4. I need to fetch the data exact matched record. When I am using LIKE operator, select query is returning two rows, but I need exact matched data record only.
Can anybody please help me with this?
SELECT *
FROM Table1
WHERE StringValue like '%4%'
But that returns two rows - both ID 1 and 2.
My expectation is I need to get ID = 1 row only
Storing delimited data like this is a well documented anti-pattern, violates basic normalisation principles and prevents the database engine from fully utilising an index.
What you can do is delimit your search value and also ensure the expression to search is correctly delimited; this is an unsargable expression however and the strorage engine will have to scan all rows every time -
declare #valueToFind varchar(10) = '4';
select *
from t
where Concat(',', t.StringValue, ',') like Concat('%,' #valueToFind, ',%');
for SQL Server 2016 and later you can use STRING_SPLIT or earlier version of SQL Server, there are many alternative, just do a search for it.
Or, you can simply do
SELECT * FROM Table1 where ',' + StringValue + ',' like '%,4,%'
How can I add a new key/val pair in an already existing JSON col in bigqyery using SQL (big query flavor).
To something like
BigQuery provides Data Manipulation Language (DML) statements such as the SQL Update statement. See:
https://cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax#update_statement
What you will have to do is retrieve the original value of your structured column and then perform a SQL UPDATE statement to set the new value of the column to be the absolute new value that you want.
Take care to realize that BigQuery is an OLAP database and is optimized for queries rather that updates or deletes. Make sure you read the information on using DML statements in BigQuery found here.
https://cloud.google.com/bigquery/docs/reference/standard-sql/data-manipulation-language
I feel like this question is less about how to update the table, but more about how to adjust existing json with extra/new key:value (then to either update table or just simply select out)
So, I assume you have table like below
and you might have another table with those new key:value pairs to use
in case if you don't really have second table - you can just use CTE like below
with new_key_val as (
select 1 id, '{"key3":"value3"}' add_json union all
select 2 id, '{"key14":"value14"}'
)
So, having above - you can use below approach
select *,
( select '{' || string_agg(trim(kv)) || ',' || trim(add_json, '{}') || '}'
from unnest(split(trim(json_col, '{}'), ',')) kv
) adjusted_json
from your_table
left join new_key_val
using(id)
with output
BigQuery supports JSON as a native data type but only offers a limited set of JSON functions. Unless your json data has a pre-defined, simple schema with known keys, you probably want to go the string-manipulation way.
i have a table column looks like below.
what is the sql query statement i can use to have multiple partial match conditions?
search by ID or Name
if search abc then list the row A1 , row A2
if search test then list the row A1 , row A2, row 3
if search ghj then list the row A2
i was trying this but nothing return:
SELECT * FROM table where colB LIKE '"ID":"%abc%"'
updating data in text
{"ItemId":"123","IDs":[{"ID":"abc","CodingSystem":"cs1"}],"Name":"test itemgh"}
{"ItemId":"123","IDs":[{"ID":"ghj","CodingSystem":"cs1"}],"Name":"test abc"}
{"ItemId":"123","IDs":[{"ID":"defg","CodingSystem":"cs1"}],"Name":"test 111"}
JSON parsing
Oracle
Looked into the JSON parsing capabilities of Oracle and I managed to make running a query like this:
select * from table t where json_exists(t.colB, '$.IDs[?(#.ID=="abc")]') or json_exists(t.colB, '$.IDs?(#.name=="abc"')
And inside the same JSON query expression:
select * from table t where json_exists(t.colB, '$.IDs[?(#.ID=="abc" || #.name=="abc")]')
The call of function json_exists() is the key to this.
The first parameter can be a VARCHAR2, and I also tried with a BLOB containing text, and it works.
The second parameter is the path to your json object attribute that needs to be tested, with the condition.
I wrote two ORed conditions for the ID and for the Name, but maybe there is a better JSON query expression you can use to include them both.
More information about json_exists() function here.
Postgres
There is a JSON datatype in Postgres that supports parsing in queries.
So, if your colB column is declared as JSON you can do something like this:
select * from table where colB->>'Name' LIKE '%abc%';
And in order to have available the array elements of the IDs array, you should use the function json_array_elements().
select * from table, json_array_elements(colB->'IDs') e where colB->>'Name' LIKE '%abc%' or e->>'ID' = 'abc';
Check an example I created for you here.
Here is an online tool for online testing your JSON queries.
Check also this question in SO.
MSSQL Server 2017
I made a couple of tests also with MS SQL Server, and I managed to create an example searching for partial matching in the name field.
select * from table where JSON_VALUE(colB,'$.Name') LIKE '%abc%';
And finally I arrived to a working query that does partial match to the Name field and full match to the ID field like this:
select * from table t
CROSS APPLY OPENJSON(colB, '$.IDs') WITH (
ID VARCHAR(10),
CodingSystem VARCHAR(10)
) e
where JSON_VALUE(t.colB,'$.Name') LIKE '%abc%'
or e.ID = 'abc';
The problem is that we need to open the IDs array, and make something like a table from it, that can be queried also by accessing its columns.
The example I created is here.
LIKE text query
Your tries are good but you misplace the % symbols. They have to be first and last in your given string:
If you want the ID to be the given value:
SELECT * FROM table where colB LIKE '%"ID":"abc"%'
If the given value can be anywhere, then don't put the "ID" part:
SELECT * FROM table where colB LIKE '%abc%'
If the given value can be only on the ID or Name field then:
SELECT * FROM table where colB LIKE '%"ID":"abc"%' OR colB LIKE '%"Name":"abc"%'
And because you are giving hard-coded identifiers of fields (eg ID and Name) that can be in variable case:
SELECT * FROM table where lower(colB) LIKE '%"id":"abc"%' OR lower(colB) LIKE '%"name":"abc"%'
Assuming that the number of spaces do not vary between the : character and the value or the name of the properties.
For partial matching you can use more % in between like '%"name":"%abc%"%':
SELECT * FROM table where lower(colB) LIKE '%"id":"abc"%' OR lower(colB) LIKE '%"name":"%abc%"%'
Regular Expressions
A different option would be to test with regular expressions.
Consider checking this: Oracle extract json fields using regular expression with oracle regexp_substr
How can i filter inside a json file in SQL server?
i have a Column call details.
{"test","source":"web"}
i want to filter by source
what i did:
select * from TABLE_NAME
CROSS APPLY OPENJSON(details,'$.source')
where value ='web'
As per Zohar's comment, make your json valid, then something like:
--{"mode":"test","source":"web"}
select * from TABLE_NAME
CROSS APPLY
OPENJSON(details)
WITH (
m varchar(256) '$.mode',
s varchar(256) '$.source'
) j
where
j.w = 'web'
But it might suit you better/simpler to just use JSON_VALUE:
select * from TABLE_NAME
WHERE json_value(details, '$.source') = 'web'
Use CROSS APPLY OPENJSON if you want to turn each row's json into a pseudotable looking like the table spec in the WITH clause. SQLServer behaves as if all the matching "rows" in each row's json are compounded into the psseudotable and auto-joined to the source data table based on where each bunch of json pseudorows came from
Use JSON_VALUE if you only really want one value out of the json and can uniquely identify a single "row" in the json from which to get the value.. Either the json only has one "row" / is not a collection, or you want a "row" out of a json collection that can be referenced according to a formula
I have a something like this in my table column:
{"InputDirection":0,"Mask":"AA","FormatString":null,"AutoCompleteValue":null,
"Filtered":"0123456789","AutoComplete":false,"ReadOnly":true}
What I want to do is to change A to N in "Mask":"AA" and remove "Filtered":"0123456789" if they exist. Mask could be in different forms like A9A, 'AAAA`, etc.
If it was in C# I could do it by myself by parsing it to JSON, etc but I need to do it within SQL.
I've found this article which shows how to parse JSON to Table. This gave me an idea that I can parse each field to temp table and make the changes on that and convert it back to JSON so update the actual field where I take this JSON field from. However, this looks like a cumbersome process for both me and the server.
Any better ideas?
You can use this LINK .
And then use the following code
select * into #demo from
(Select * from parseJSON('{"InputDirection":0,"Mask":"AA","FormatString":null,"AutoCompleteValue":null,
"Filtered":"0123456789","AutoComplete":false,"ReadOnly":true}
')) a
select * from #demo
--- CHANGE THE DATA HERE AS REQUIRED
DECLARE #MyHierarchy JSONHierarchy;
INSERT INTO #myHierarchy
select * from #demo;
-- USE THIS VALUE AND UPDATE YOUR JSON COLUMN
SELECT dbo.ToJSON(#MyHierarchy)
drop table #demo
I may be getting something wrong here but why can’t you simply use REPLACE to update what’s needed and LIKE to identify JSON strings that should be updated?
update table_T
set json_string = REPLACE(json_string, '"Filtered":"0123456789",', '')
where json_string like '%"Mask":"AA"%'
Not sure I understand why do you need to parse it….