Not sure what's wrong with this below SQL query. I am getting below error message
'Incorrect syntax near the keyword 'with'. If this statement is a common table expression, an xmlnamespaces clause or a change tracking context clause, the previous statement must be terminated with a semicolon.'
Can anyone please help me to fix this query
DECLARE #json NVARCHAR(4000) = N'{
"pets" : {
"cats" : [
{ "id" : 1, "name" : "Fluffy", "gender" : "Female" },
{ "id" : 2, "name" : "Long Tail", "gender" : "Female" },
{ "id" : 3, "name" : "Scratch", "gender" : "Male" }
],
"dogs" : [
{ "name" : "Fetch", "gender" : "Male" },
{ "name" : "Fluffy", "gender" : "Male" },
{ "name" : "Wag", "gender" : "Female" }
]
}
}'
SELECT * FROM OPENJSON(#json, '$.pets.cats')
WITH(
[Cat Id] int,
[Cat Name] varchar(60),
[Gender] varchar(6),
[Cats] nvarchar(max)
);
P.S. I am using SQL server 2017 14.0
Related
DECLARE #json NVARCHAR(4000) = N'{
"pets" : {
"cats" : [
{ "id" : 1, "name" : "Fluffy", "sex" : "Female" },
{ "id" : 2, "name" : "Long Tail", "sex" : "Female" },
{ "id" : 3, "name" : "Scratch", "sex" : "Male" }
]
}
}'
SELECT * FROM OPENJSON(#json, '$.pets.cats') WITH --we have the "main" json address here
(
id INT '$.id', --sub-address
name varchar(10) '$.name', --sub-address
sex varchar(10) '$.sex' --sub-address
)
The results are:
id
name
sex
1
Fluffy
Female
2
Long Tail
Female
3
Scratch
Male
I want to include another column which will not depend on the JSON, but be a custom specified string ("mammal"), so that the output is like this:
id
name
sex
Type
1
Fluffy
Female
mammal
2
Long Tail
Female
mammal
3
Scratch
Male
mammal
How would I achieve this?
You can simply add a constant in your SELECT statement:
DECLARE #json NVARCHAR(4000) = N'{
"pets" : {
"cats" : [
{ "id" : 1, "name" : "Fluffy", "sex" : "Female" },
{ "id" : 2, "name" : "Long Tail", "sex" : "Female" },
{ "id" : 3, "name" : "Scratch", "sex" : "Male" }
]
}
}'
SELECT *, 'mammal' AS Type FROM OPENJSON(#json, '$.pets.cats') WITH --we have the "main" json address here
(
id INT '$.id', --sub-address
name varchar(10) '$.name', --sub-address
sex varchar(10) '$.sex' --sub-address
)
I have a table in my database as follows
my_table:jsonb
[ {
"name" : "world map",
"type" : "activated",
"map" : [ {
"displayOrder" : 0,
"value" : 123
}, {
"displayOrder" : 1,
"value" : 456
}, {
"displayOrder" : 2,
"value" : 789
} ]
}, {
"name" : "regional map",
"type" : "disabled"
} ]
I would like to create indices for the name, type and displayOrder fields, which would be the best way?
I am trying to create a BQ Table from AVRO file. I am getting this error when i run the BQ load job:
"Error while reading data, error message: The Apache Avro library
failed to parse the header with the following error: Unexpected type
for default value. Expected long, but found null: null"
The Schema of the AVRO file is:
{
"type" : "record",
"name" : "Pair",
"namespace" : "org.apache.avro.mapred",
"fields" : [ {
"name" : "key",
"type" : "int",
"doc" : ""
}, {
"name" : "value",
"type" : {
"type" : "record",
"name" : "CustomerInventoryOrderItems",
"namespace" : "com.test.customer.order",
"fields" : [ {
"name" : "updated_at",
"type" : "long"
}, {
"name" : "inventory_order_items",
"type" : {
"type" : "map",
"values" : {
"type" : "array",
"items" : {
"type" : "record",
"name" : "CustomerInventoryOrderItem",
"fields" : [ {
"name" : "order_item_id",
"type" : "int",
"default" : null
}, {
"name" : "updated_at",
"type" : "long"
}, {
"name" : "created_at",
"type" : "long"
}, {
"name" : "product_id",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "type_id",
"type" : "int",
"default" : null
}, {
"name" : "event_id",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "price",
"type" : [ "null", "double" ],
"default" : null
}, {
"name" : "tags",
"type" : [ "null", "string" ],
"default" : null
}, {
"name" : "estimated_ship_date",
"type" : [ "null", "long" ],
"default" : null
} ]
}
}
}
} ]
},
"doc" : "",
"order" : "ignore"
} ]
}
I am not sure what is wrong with the schema or anything else, because of which I am unable to load the data.
The problem is most likely the fields that have type int but you have null as the default value. For example:
"name" : "type_id",
"type" : "int",
"default" : null
The default should either be changed to be an integer or the type should be changed to be a union that includes null (like many of the other fields).
We are using Postgres DB , in that we have one table contains column of type JSON , format is like below
{
"name" : "XXX",
"id" : "123",
"course" :[
{
"name" : "java",
"tutor":"YYYY"
},
{
"name" : "python",
"tutor":"ZZZZ"
}
]
}
{
"name" : "XXX",
"id" : "123",
"course" :[
{
"name" : "java",
"tutor":"YYYY"
},
{
"name" : "python",
"tutor":"ZZZZ"
}
]
}
like this for example we have two rows , and in the json column we have each like above
i want to Postgre query , which will check the number of elements in the course array and if it is more than one , then only return that row
am not getting how to count the array elements from inside the json key
can any please suggest
why not just json_array_length?.. eg:
f=# with c(j) as (values('{
"name" : "XXX",
"id" : "123",
"course" :[
{
"name" : "java",
"tutor":"YYYY"
},
{
"name" : "python",
"tutor":"ZZZZ"
}
]
}'::json))
select json_array_length(j->'course') from c;
json_array_length
-------------------
2
(1 row)
so smth like
select * from table_name where json_array_length(j->'course') > 1
I work on a Azure HDInsight cluster with version 3.6. It uses Hortonworks HDP 2.6, which comes with Hive 2.1.0 (on Tez 0.8.4).
I have some internal Hive tables with nested struct fields stored in Avro format. Here is one example CREATE statement:
CREATE TABLE my_example_table(
some_field STRING,
some_other_field STRING,
some_struct struct<field1: BIGINT, inner_struct struct<field2: STRING, field3: STRING>>)
PARTITIONED BY (year INT, month INT)
STORED AS AVRO;
I populate these tables with from an external table which also is stored as avro, like this:
INSERT INTO TABLE my_example_table
PARTITION (year, month)
SELECT ....
FROM my_external_table;
When I want to query the internal tables I got the following error: Failed with exception java.io.IOException:org.apache.avro.AvroTypeException: Found core.record_0, expecting union
I extracted the avro schema from one of these internal tables with the Avro tools and recognized that Hive creates union types from the structs I defined:
{
"type" : "record",
"name" : "my_example_table",
"namespace" : "my_namespace",
"fields" : [ {
"name" : "some_field",
"type" : [ "null", "string" ],
"default" : null
}, {
"name" : "some_other_field",
"type" : [ "null", "string" ],
"default" : null
}, {
"name" : "my_struct",
"type" : [ "null", {
"type" : "record",
"name" : "record_0",
"namespace" : "",
"doc" : "struct<field1: BIGINT, struct<field2: STRING, field3: STRING>>",
"fields" : [ {
"name" : "field1",
"type" : [ "null", "long" ],
"doc" : "bigint",
"default" : null
}, {
"name" : "inner_struct",
"type" : [ "null", {
"type" : "record",
"name" : "record_2",
"namespace" : "",
"doc" : "struct<field2: STRING, field3: STRING>",
"fields" : [ {
"name" : "field2",
"type" : [ "null", "string" ],
"doc" : "bigint",
"default" : null
}, {
"name" : "field2",
"type" : [ "null", "long" ],
"doc" : "bigint",
"default" : null
}]
}
]}
]}
]}
}
Whats going wrong here? I'm pretty sure exactly this worked some days ago, so I conjectured that Microsoft switched to another patch version of HDP for HDInsight clusters which has another Avro or Hive version, but I haven't found any indications to that.
I found this: https://issues.apache.org/jira/browse/HIVE-15316 which seems to be pretty similar problem (on the same Hive version).
Does anybody know whats going wrong here and what I could do to fix this problem or as a workaround?