i'm pretty new here and i'm trying to parse my JSON but keep getting this error. i have the below query and every time i try running it i get an error, any chance i can get some help? My intended result will be for the data to be displayed on a normal sql grid and later to insert the data into a table with the below formatting.
Declare #json varchar(MAX) = '
{"stops":
{
"__type": "stop",
"__name": "stops",
"company_id": "xxx",
"actual_arrival": "20210910130000-0500",
"actual_departure": "20210910140000-0500",
"address": "123 noneyabizwax",
"appt_required": false,
"city_id": 144537,
"city_name": "LEWISBURG",
"state": "TN",
"zip_code": "37091",
"confirmed": false,
"driver_load_unload": "N",
"eta": "20210910122900-0500",
"eta_distance": 46,
"eta_late": false,
"eta_oor": false,
"order_id": "12345",
"planned_arrival_time": "20210910122900-0500",
"sched_arrive_early": "20210910130000-0500",
"status": "D",
"stop_type": "PU"
}';
select *
from OPENJSON ( #json )
With (
Type Varchar(50) '$stops.__type',
Name Varchar(50) '$stops.__name',
CompID Varchar(100) '$stops.company_id' ,
AcArrvl DATETIME '$stops.actual_arrival' ,
AcDprtr DATETIME '$stops.actual_departure',
Addrs Varchar (200) '$stops.address',
Appt Varchar(50) '$stops.appt_required',
CityID Varchar(50) '$stops.city_id',
City Varchar(200) '$stops.city_name',
State nvarchar(max) '$stops.state',
Zip int '$stops.zip_code',
Confirmed Varchar(50) '$stops.confirmed',
DrvrLd nvarchar(max) '$stops.driver_load_unload',
ETA DATETIME '$stops.eta',
ETADISTNC INT '$stops.eta_distance',
ETALate nvarchar(max) '$stops.eta_late',
ETAOOR nvarchar(max) '$stops.eta_oor',
OrderID INT '$stops.order_id',
PlnndArrvlTm DATETIME '$stops.planned_arrival_time',
SchdArrvEarly DATETIME '$stops.sched_arrive_early',
Status nvarchar(max) '$stops.status',
StopType Varchar(50) '$stops.stop_type',
[Stops] nvarchar(max) AS JSON
)
Declare #json varchar(MAX) = '
{"stops":
{
"__type": "stop",
"__name": "stops",
"company_id": "xxx",
"actual_arrival": "20210910130000-0500",
"actual_departure": "20210910140000-0500",
"address": "123 noneyabizwax",
"appt_required": false,
"city_id": 144537,
"city_name": "LEWISBURG",
"state": "TN",
"zip_code": "37091",
"confirmed": false,
"driver_load_unload": "N",
"eta": "20210910122900-0500",
"eta_distance": 46,
"eta_late": false,
"eta_oor": false,
"order_id": "12345",
"planned_arrival_time": "20210910122900-0500",
"sched_arrive_early": "20210910130000-0500",
"status": "D",
"stop_type": "PU"}
}';
WITH date_convert
AS
(
SELECT CONVERT(DATETIMEOFFSET,STUFF(STUFF(STUFF(STUFF(STUFF(STUFF(JSON_VALUE(#JSON, '$.stops.actual_arrival'),18,0,':'),13,0,':'),11,0,':'),9,0,' '),7,0,'-'),5,0,'-')) AcArrvl,
CONVERT(DATETIMEOFFSET,STUFF(STUFF(STUFF(STUFF(STUFF(STUFF(JSON_VALUE(#JSON, '$.stops.actual_departure'),18,0,':'),13,0,':'),11,0,':'),9,0,' '),7,0,'-'),5,0,'-')) AcDprtr,
CONVERT(DATETIMEOFFSET,STUFF(STUFF(STUFF(STUFF(STUFF(STUFF(JSON_VALUE(#JSON, '$.stops.eta'),18,0,':'),13,0,':'),11,0,':'),9,0,' '),7,0,'-'),5,0,'-')) ETA,
CONVERT(DATETIMEOFFSET,STUFF(STUFF(STUFF(STUFF(STUFF(STUFF(JSON_VALUE(#JSON, '$.stops.planned_arrival_time'),18,0,':'),13,0,':'),11,0,':'),9,0,' '),7,0,'-'),5,0,'-')) PlnndArrvlTm,
CONVERT(DATETIMEOFFSET,STUFF(STUFF(STUFF(STUFF(STUFF(STUFF(JSON_VALUE(#JSON, '$.stops.sched_arrive_early'),18,0,':'),13,0,':'),11,0,':'),9,0,' '),7,0,'-'),5,0,'-')) SchdArrvEarly
)
select Type, Name,CompID,date_convert.AcArrvl,date_convert.AcDprtr,Addrs,Appt,CityID,City,State,Zip,Confirmed,DrvrLd,date_convert.ETA,ETADISTNC,ETALate,ETAOOR,OrderID,date_convert.PlnndArrvlTm,
date_convert.SchdArrvEarly, Status,StopType
from date_convert,
OPENJSON (#json)
With (
Type Varchar(50) '$.stops.__type',
Name Varchar(50) '$.stops.__name',
CompID Varchar(100) '$.stops.company_id' ,
Addrs Varchar (200) '$.stops.address',
Appt Varchar(50) '$.stops.appt_required',
CityID Varchar(50) '$.stops.city_id',
City Varchar(200) '$.stops.city_name',
State nvarchar(max) '$.stops.state',
Zip int '$.stops.zip_code',
Confirmed Varchar(50) '$.stops.confirmed',
DrvrLd nvarchar(max) '$.stops.driver_load_unload',
ETADISTNC INT '$.stops.eta_distance',
ETALate nvarchar(max) '$.stops.eta_late',
ETAOOR nvarchar(max) '$.stops.eta_oor',
OrderID INT '$.stops.order_id',
Status nvarchar(max) '$.stops.status',
StopType Varchar(50) '$.stops.stop_type'
)
It looks a little complicated but I've included an explanation below:
Add } (curly bracket) at the end of this line "stop_type": "PU"
You must add . (dot) after every $ sign in your With clause.
Then we need to remove [Stops] nvarchar(max) AS JSON line because you don't need to open another JSON object and also don't forget to remove a ,(comma) after StopType Varchar(50) '$.stops.stop_type,'
We need to convert your 5 date strings into at least a partially human readable format. To do this we need to use CONVERT() with STUFF() function. I could have converted all 5 strings in SELECT statment but because it would looks ugly I decided to do this in CTE (I named CTE as a date_convert)
Because we store our converted dates in CTE we don't need them in your With clause. So we delete them.
In order to see our converted dates with the rest of columns in one table we need to add CTE's name into FROM clause. After it's done we have to add name of each column both from your With clause and from CTE to SELECT statment. The order is not important but I've preserved the order from your original question.
https://dbfiddle.uk/?rdbms=sqlserver_2016&fiddle=5b3915d8df86127e1f3c84d0a9880e31
Related
I think the problem is because I have hard brackets [] in the 'groups' object.
JSON:
declare #json nvarchar(max) = N'{ "agents": [ {
"id": 9544321,
"uuid":"xxxx-xxx-xxxx",
"groups": [
{
"name": "GROUP NAME HERE",
"id": 123456
}
],
"support": true
}]}'
What I'm trying to get to is:
id
uuid
group name
group id
9544321
xxxx-xxx-xxxx
GROUP NAME HERE
123456
And then if there is a second group specified in the 'groups' object that'd be two rows. etc
This is my code so far, but it just leaves the [group name] and [group id] columns blank. I've also tried a cross apply and that didn't work (same result).
select *
from openjson(#json, '$.agents')
with (
id int '$.id',
uuid varchar(60) '$.uuid',
groups nvarchar(max) '$.groups' as json
)
outer apply openjson (groups)
with ([group name] nvarchar(max) '$',
[group id] int '$'
)
Try this:
select *
from openjson(#json, '$.agents')
with (
id int '$.id',
uuid varchar(60) '$.uuid',
groups nvarchar(max) '$.groups' as json
)
outer apply openjson (groups, '$')
with ([name] nvarchar(max),
[id] int
)
Did you find the difference? You have placed the value 'Group Name' which is the value, instead of keys in json.
#BeckyG, you're almost there. Please, pay attention to lines 29 and 30:
DECLARE #json NVARCHAR(MAX) = N'{ "agents": [ {
"id": 9544321,
"uuid":"xxxx-xxx-xxxx",
"groups": [
{
"name": "GROUP NAME HERE",
"id": 123456
},
{
"name": "GROUP NAME HERE 2",
"id": 1234567
}
],
"support": true
}]}';
SELECT agents.id,
agents.uuid,
groups.name [group name],
groups.id [group id]
FROM OPENJSON(#json, '$.agents')
WITH (
id INT '$.id',
uuid VARCHAR(60) '$.uuid',
groups NVARCHAR(MAX) '$.groups' AS JSON
) AS agents
OUTER APPLY OPENJSON (agents.groups)
WITH (
[name] NVARCHAR(MAX) '$.name',
[id] INT '$.id'
) AS groups;
Happy coding!
I need to import json data into an SQL table however there are some fields that I only want the numeric value from the list. Is there way to do a nested select using openjson?
I'm calling a stored procedure on a Microsoft SQL 2016 server from pyton and passing the json as a parameter.
Select * from OpenJson(#json)
WITH (id int 'strict $.id',
[Name] nvarchar(50) '$.name',
term_id nvarchar(max) as json,
phone nvarchar(20) '$.phone',
email nvarchar(50) '$.email',
street nvarchar(75) '$.street',
street2 nvarchar(75) '$.street2',
city nvarchar(50) '$.city',
zip nvarchar(20),
country_id nvarchar(max) as JSON)
The table has the following fields
ID, Name, Term_ID, Phone, Email, Street, Street2, Zip, City, Country_ID
see the json below. On the fields term_id and country_id I only want to pull the numeric value not the string.
[{
"id": 340,
"name": Joe,
"term_id": [
3,
"Net 30"
],
"phone": 123-456-7890,
"email": false,
"street": "Some Street",
"street2": false,
"zip": "01234",
"city": "SomeCity",
"country_id": [
1,
"United States"
]
}]
Struggling to find an answer to the below JSON problem. I would like to display the entire SKU/Quantity list in the "shipmentItems" array to their respective column with a comma-separated value. My example below only allows me to display the first SKU/quantity from the array but, my goal is to get all listed in the columns with comma-separated.
JSON example:
{"shipments": [
{
"shipmentId": 100003768,
"orderNumber": "9648219086",
"shipDate": "2021-10-28",
"serviceCode": "ups_ground",
"shipmentItems": [
{
"orderItemId": 1464643208,
"lineItemKey": "10322938560608",
"sku": "SCPXTSS-BAG-06",
"name": "SCOOP-PLATE-06 (1000ml)",
"weight": {
"value": 0,
"units": "ounces",
"WeightUnits": 1
},
"quantity": 1,
"unitPrice": 0,
"taxAmount": null
},
{
"orderItemId": 1464643207,
"lineItemKey": "10322938527840",
"sku": "SCPZRTS-TRAY-01",
"name": "Beef: Tray 3 (Fill 004)<br>",
"weight": {
"value": 60,
"units": "ounces",
"WeightUnits": 1
},
"quantity": 1,
"unitPrice": 102.72,
"taxAmount": null
}
],
"labelData": null,
"formData": null
}
]
}
SQL query I'm using:
DECLARE #JSON varchar(max)
SELECT #JSON = BulkColumn
FROM OPENROWSET (BULK 'C:\Users\XPS-LT\json\today\shipments_20211031.json', SINGLE_CLOB)
IMPORT
SELECT *
FROM OPENJSON (#JSON, '$.shipments')
WITH
(
[shipmentId] bigint,
[orderNumber] nvarchar(60),
[shipDate] date,
[serviceCode] nvarchar(30),
[sku] nvarchar(MAX) N'$.shipmentItems[0].sku',
[quantity] int N'$.shipmentItems[0].quantity'
)
;
The "shipmentItems" part of the input JSON is an array, so you need an AS JSON clause in the first explicit schema and an additional OPENJSON() call:
DECLARE #json nvarchar(max)
...
SELECT
j.[shipmentId], j.[orderNumber], j.[shipDate], j.[serviceCode],
a.[sku], a.[quantity]
FROM OPENJSON (#json, '$.shipments') WITH (
[shipmentId] bigint,
[orderNumber] nvarchar(60),
[shipDate] date,
[serviceCode] nvarchar(30),
[shipmentItems] nvarchar(max) AS JSON
) j
OUTER APPLY (
SELECT
STRING_AGG([sku], ',') WITHIN GROUP (ORDER BY [orderItemId]),
STRING_AGG([quantity], ',') WITHIN GROUP (ORDER BY [orderItemId])
FROM OPENJSON (j.shipmentItems) WITH (
[orderItemId] int '$.orderItemId',
[sku] nvarchar(max) '$.sku',
[quantity] int N'$.quantity'
)
) a ([sku], [quantity])
Result:
shipmentId orderNumber shipDate serviceCode sku quantity
100003768 9648219086 2021-10-28 ups_ground SCPZRTS-TRAY-01,SCPXTSS-BAG-06 1,1
I'm new to JSON methods in SQL
I'm working with a huge JSON field, which has such a structure :
{
"A": 1,
"B": 2,
"C": 0,
"data": [
{
"id": "id_i_want",
"value": "[{
"prop1":7,
"prop2":"X",
"prop3":"4",
"passages":[
{
"id":0,
"number":"E24",
"date":"11/12/2019"
},
{
"id":0,
"number":"F28",
"date":"11/11/2019"
},
{
...
}
]
}]
},
{
"id": "id_i_do_NOT_want",
"value": Same structure as above
}
]
}
This JSON fields is stored in a nvarchar(MAX) field in SQLServer.
So the JSON has a property data, which contains a list of elements.
These elements have a value property, which contains a list of passages.
All the passages currently have id = 0
What I need to do :
I would like ton increment all the id of passages, starting from 1, but only the ones in the object which has the ID id_i_want, and NOT the others.
How can I do that with a SQL script ?
I tried to follow this post, but without success
Any help appreciated
First change the Json data to tabular data, then update the table, and then convert the table back to Json.
I have prepared the following code, you can use it easily with a little change.
declare #varData nvarchar(max) = '{
"A": 1,
"B": 2,
"C": 0,
"data": [
{
"id": "id_i_want",
"value": [{
"prop1":7,
"prop2":"X",
"prop3":"4",
"passages":[
{
"id":0,
"number":"E24",
"date":"11/12/2019"
},
{
"id":0,
"number":"F28",
"date":"11/11/2019"
}
]
}]
},
{
"id": "id_i_do_NOT_want"
}
]
}';
DECLARE #jsontable TABLE (A varchar(5), b varchar(5), c varchar(5),id NVARCHAR(50),prop1 int,prop2 varchar(5),prop3 varchar(5),mid int ,number varchar(5),date date);
DECLARE #maintable TABLE (A varchar(5), b varchar(5), c varchar(5),id NVARCHAR(50),prop1 int,prop2 varchar(5),prop3 varchar(5),mid int ,number varchar(5),date date);
insert into #jsontable
SELECT A,b,C,id,prop1,prop2,prop3,mid,number,date
FROM OPENJSON(#varData)
WITH (
A varchar(5) '$.A',
B varchar(5) '$.B',
C varchar(5) '$.C',
jdata NVARCHAR(MAX) '$.data' AS JSON
)
OUTER APPLY OPENJSON(jdata)
WITH (
id NVARCHAR(50) '$.id',
jvalue NVARCHAR(MAX) '$.value' AS JSON
)
OUTER APPLY OPENJSON(jvalue)
WITH (
prop1 int '$.prop1',
prop2 varchar(5) '$.prop2',
prop3 varchar(5) '$.prop3',
jpassages NVARCHAR(MAX) '$.passages' AS JSON
)
OUTER APPLY OPENJSON(jpassages)
WITH (
mid int '$.id',
number varchar(5) '$.number',
date date '$.date'
)
where id = 'id_i_want'
Declare #A varchar(5)
Declare #b varchar(5)
Declare #c varchar(5)
Declare #id NVARCHAR(50)
Declare #prop1 int
Declare #prop2 varchar(5)
Declare #prop3 varchar(5)
Declare #mid int = 0
Declare #number varchar(5)
Declare #date date
While((Select Count(*) From #jsontable)>0)
Begin
set #mid = #mid + 1
Set #A=(Select Top 1 A From #jsontable)
Set #B=(Select Top 1 B From #jsontable)
Set #C=(Select Top 1 C From #jsontable)
Set #id=(Select Top 1 id From #jsontable)
Set #prop1=(Select Top 1 prop1 From #jsontable)
Set #prop2=(Select Top 1 prop2 From #jsontable)
Set #prop3=(Select Top 1 prop3 From #jsontable)
Set #number=(Select Top 1 number From #jsontable)
Set #date=(Select Top 1 date From #jsontable)
insert into #maintable values
(#A,#B,#C,#id,#prop1,#prop2,#prop3,#mid,#number,#date)
Delete #jsontable Where A=#A and B = #B and C = #C and id = #id and prop1 = #prop1
and prop2 = #prop2 and prop3 = #prop3 and number = #number and date = #date
End
select * from #maintable
demo in db<>fiddle
I'm sifting through a JSON movie database and can get most of the data without a problem. For each film, I've put the raw JSON in an nvarchar(max) column called jsondata, then added other columns to populate for indexing and quick retrieval purposes.
With my UPDATE statement, I'm able to populate all columns except for one, in which I need to parse through and find an element.
Here's an example of the JSON:
{
"title": "Amnesia Love",
"genres": [{
"id": 35,
"name": "Comedy"
}],
"id": 508989,
"original_language": "tl",
"overview": "A guy is trying to discover his true identity after being found unconscious in a remote island.",
"popularity": 2.583,
"release_date": "2018-02-28",
"credits": {
"cast": [{
"cast_id": 2,
"character": "",
"credit_id": "5a9af6ecc3a3680b7d024627",
"gender": 0,
"id": 1230955,
"name": "Paolo Ballesteros",
"order": 1,
"profile_path": "/8Cey11JPMWBCGuIcKBXvb6OQ7Je.jpg"
},
{
"cast_id": 3,
"character": "",
"credit_id": "5a9af6f3c3a3680b57024465",
"gender": 0,
"id": 1166094,
"name": "Yam Concepcion",
"order": 2,
"profile_path": "/fiPaXTkq440VHXDqlMRHtOpoLWT.jpg"
}],
"crew": [{
"credit_id": "5a9af6cc0e0a260649024c6a",
"department": "Directing",
"gender": 0,
"id": 1989658,
"job": "Director",
"name": "Albert Langitan",
"profile_path": null
},
{
"credit_id": "5a9af6dec3a3680b2d01f152",
"department": "Writing",
"gender": 0,
"id": 1989658,
"job": "Screenplay",
"name": "Albert Langitan",
"profile_path": null
}]
},
}
Here's the table schema:
CREATE TABLE dbo.moviedb(
id int IDENTITY(1,1) NOT NULL,
jsondata nvarchar(max) NULL,
title nvarchar(200) NULL,
movie_id varchar(255) NULL,
original_language char(2) NULL,
overview nvarchar(1000) NULL,
popularity float NULL,
release_date datetime NULL,
genre nvarchar(100) NULL,
director nvarchar(100)
)
Here is an update statement to populate the columns:
UPDATE t
SET t.title = j.title, t.movie_id = j.id, t.original_language = j.original_language,
t.overview = j.overview, t.popularity = j.popularity, t.release_date = j.release_date,
t.genre = ISNULL(JSON_VALUE(t.jsondata,'$.genres[0].name'),''),
t.director = JSON_VALUE(t.jsondata,'$.credits.crew[0].name')
FROM tmdb t
CROSS APPLY OPENJSON(jsondata)
WITH(title nvarchar(200), id int, original_language char(2), overview nvarchar(max),
popularity float, release_date datetime,
genres nvarchar(max) as JSON,
credits nvarchar(max) as JSON
) AS j
My problem is getting the director name. Since within $.credits, there are cast and crew elements, and within those, potentially many entries each - I'm not clear how to parse through via JSON_QUERY or using CROSS APPLY to basically say, 'find the $.credits.crew.job = "Director", then give me the $.credits.crew.name'.
Ok, after the comments, I think I've got it now. I use OUTER APPLY for the Director, since I still want to get the movie info even if no Director is specified. This lead me to one other issue - when there was more than 1 Director on the movie. But I think I can deal with that.
UPDATE t
SET t.title = j.title, t.movie_id = j.id, t.original_language = j.original_language,
t.overview = j.overview, t.popularity = j.popularity, t.release_date = j.release_date,
t.genre = ISNULL(JSON_VALUE(t.jsondata,'$.genres[0].name'),''),
t.director = k.name
FROM tmdb t
CROSS APPLY OPENJSON(jsondata)
WITH (title nvarchar(200), id int, original_language char(2),
overview nvarchar(max), popularity float, release_date datetime,
credits nvarchar(max) as JSON
) AS j
OUTER APPLY OPENJSON(j.credits,'$.crew')
WITH (job nvarchar(50), name nvarchar(100)) AS k
WHERE k.job = 'Director' OR k.job IS NULL