Here's the issue. I have a column in my database (type nvarchar(max)) that I am storing JSON in. I am storing both plain strings or objects in that column like the following:
JsonTable
|--|-------------------|
|Id|JsonValue |
|--|-------------------|
|0 |{"sample":"object"}|
|--|-------------------|
|1 |"plain-string" |
|--|-------------------|
I am trying to use JSON_MODIFY to merge these values with another table's values.
The following works fine for just objects, but not strings:
SELECT JSON_MODIFY('{}', '$.Values', JSON_QUERY(JsonValue))
FROM JsonTable
WHERE Id = 0 -- Fails when string is included in results
-- Result = |------------------------------|
|{"Values":{"sample":"object"} |
|------------------------------|
However it fails to parse the ordinary string (understandably since it is not JSON)
So then my solution was to add a case statement to handle strings. However this does not work as wrapping it in a CASE statement string escapes the JSON_QUERY object and garbles it up in the final JSON_MODIFY result.
The following does not work as expected:
SELECT JSON_MODIFY('{}', '$.Values',
CASE
WHEN ISJSON(JsonValue) > 0 THEN JSON_QUERY(JsonValue)
ELSE REPLACE(JsonValue, '"','')
END)
FROM JsonTable
-- Result = |-------------------------------------|
|{"Values":"{\"sample\"::\"object\"}" |
|-------------------------------------|
|{"Values":"plain-string" |
|-------------------------------------|
So I was unable to figure out really why wrapping JSON_QUERY in a CASE statement doesnt return properly, but instead I started using this workaround which is a bit verbose and messy but it works perfectly fine:
SELECT
CASE
WHEN ISJSON(JsonValue) > 0
THEN
(SELECT JSON_MODIFY('{}', '$.Values', JSON_QUERY(JsonValue)))
ELSE
(SELECT JSON_MODIFY('{}', '$.Values', REPLACE(JsonValue, '"','')))
END
FROM JsonTable
-- Result = |-------------------------------------|
|{"Values":{"sample":"object"} |
|-------------------------------------|
|{"Values":"plain-string" |
|-------------------------------------|
Have you tried using the string_escape function to format your string for JSON; i.e. assuming your issue is related to the correctly escaping the quotes? http://sqlfiddle.com/#!18/9eecb/24391
SELECT JSON_MODIFY('{}', '$.Values',
case
when ISJSON(JsonValue) = 1 then JSON_QUERY(JsonValue)
else STRING_ESCAPE(JsonValue,'json')
end
)
FROM (
values
(0, '{"sample":"object"}')
,(1, 'plain-string')
,(2, '"plain-string2"')
) JsonTable(Id,JsonValue)
STRING_ESCAPE documentation
Related
I have a value in a JSON column that is sometimes all null in an Azure Databricks table. The full process to get to JSON_TABLE is: read parquet, infer schema of JSON column, convert the column from JSON string to deeply nested structure, explode any arrays within. I am working in SQL with python-defined UDFs (json_exists() checks the schema to see if the key is possible to use, json_get() gets a key from the column or returns a default) and want to do the following:
SELECT
ID, EXPLODE(json_get(JSON_COL, 'ARRAY', NULL)) AS SINGLE_ARRAY_VALUE
FROM
JSON_TABLE
WHERE
JSON_COL IS NOT NULL AND
json_exists(JSON_COL, 'ARRAY')==1
When the data has at least one instance of JSON_COL containing ARRAY, the schema is such that this has no problems. If, however, the data has all null values in JSON_COL.ARRAY, an error occurs because the column has been inferred as a string type (error received: input to function explode should be array or map type, not string). Unfortunately, while the json_exists() function returns the expected values, the error still occurs even when the returned dataset would be empty.
Can I get around this error via casting or replacement of nulls? If not, what is an alternative that still allows inferring the schema of the JSON?
Note: This is a simplified example. I am writing code to generate SQL code for hundreds of similar data structures, so while I am open to workarounds, a direct solution would be ideal. Please ask if anything is unclear.
Example table that causes error:
| ID | JSON_COL |
| 1 | {"_corrupt_record": null, "otherInfo": [{"test": 1, "from": 3}]} |
| 2 | {"_corrupt_record": null, "otherInfo": [{"test": 5, "from": 2}]} |
Example table that does not cause error:
| ID | JSON_COL |
| 1 | {"_corrupt_record": null, "array": [{"test": 1, "from": 3}]} |
| 2 | {"_corrupt_record": null, "otherInfo": [{"test": 5, "from": 2}]} |
This question seems like it might hold the answer, but I was not able to get anything working from it.
You can filter the table before calling json_get and explode, so that you only explode when json_get returns a non-null value:
SELECT
ID, EXPLODE(json_get(JSON_COL, 'ARRAY', NULL)) AS SINGLE_ARRAY_VALUE
FROM (
SELECT *
FROM JSON_TABLE
WHERE
JSON_COL IS NOT NULL AND
json_exists(JSON_COL, 'ARRAY')==1
)
I have below JSON from which i need to fetch the value of issuedIdentValue where issuedIdentType = PANCARD
{
"issuedIdent": [
{"issuedIdentType":"DriversLicense","issuedIdentValue":"9797979797979797"},
{"issuedIdentType":"SclSctyNb","issuedIdentValue":"078-01-8877"},
{"issuedIdentType":"PANCARD","issuedIdentValue":"078-01-8877"}
]
}
I can not hard-code the index value [2] in my below query as the order of these records can be changed. So want to get rid off any hardcoded index.
select json_value(
'{"issuedIdent": [{"issuedIdentType":"DriversLicense","issuedIdentValue":"9797979797979797"},{"issuedIdentType":"SclSctyNb","issuedIdentValue":"078-01-8877"}, {"issuedIdentType":"PANCARDSctyNb","issuedIdentValue":"078-01-8877"}]}',
'$.issuedIdent[2].issuedIdentValue'
) as output
from d1entzendev.ExternalEventLog
where
eventname = 'CustomerDetailsInqSVC'
and applname = 'digitalBANKING'
and requid = '4fe1fa1b-abd4-47cf-834b-858332c31618';
What changes will need to apply in json_value function to achieve the expected result
In Oracle 12c or higher, you can use JSON_TABLE() for this:
select value
from json_table(
'{"issuedIdent": [{"issuedIdentType":"DriversLicense","issuedIdentValue":"9797979797979797"},{"issuedIdentType":"SclSctyNb","issuedIdentValue":"078-01-8877"}, {"issuedIdentType":"PANCARD","issuedIdentValue":"078-01-8877"}]}',
'$.issuedIdent[*]' columns
type varchar(50) path '$.issuedIdentType',
value varchar(50) path '$.issuedIdentValue'
) t
where type = 'PANCARD'
This returns:
| VALUE |
| :---------- |
| 078-01-8877 |
I have a project which uses jooq + postgres with multiple tables and relations between them.
while I was creating a select query with jooq I had to use arrayAgg for my specific scenario.
dslContext.select(arrayAgg(tableName.INTEGER_LETS_SAY).as("static_name")
the specific column INTEGER_LETS_SAY is nullable.
when the results passed in arrayAgg are all null then the response of the postgres is '{null}' ( tested with getQuery().getSql() ) but the where statement cannot return true for all the methods I tried.
for example :
field("static_name", Long[].class).isNull()
field("static_name", Long[].class).equal(new Long[] {null})
field("static_name", Long[].class).equal(DSL.castNull(Long[].class)
field("static_name", Long[].class).cast(String.class).eq(DSL.value("{null}")))
field("static_name", Long[].class).cast(String.class).eq(DSL.value("'{null}'")))
any clue what am I doing wrong?
Note : I did try the query with plain sql and static_name = '{null}' worked
{NULL} is PostgreSQL's text representation of an array containing one SQL NULL value. You can try it like this:
select (array[null]::int[])::text ilike '{null}' as a
It yields:
a |
----|
true|
Note, I'm using ilike for case insensitive comparison. On my installation, I'm getting {NULL}, not {null}. If you wanted to compare things as text, you could do it using Field.likeIgnoreCase(). E.g. this works for me:
System.out.println(ctx.select(
field(val(new Long[] { null }).cast(String.class).likeIgnoreCase("{null}")).as("a")
).fetch());
Producing:
+----+
|a |
+----+
|true|
+----+
But much better, do not work with the text representation. Instead, follow this suggestion here. In SQL:
select true = all(select a is null from unnest(array[null]::int[]) t (a)) as a
In jOOQ:
System.out.println(ctx.select(
field(inline(true).eq(all(
select(field(field(name("a")).isNull()))
.from(unnest(val(new Long[] { null })).as("t", "a"))
))).as("a")
).fetch());
It gets a bit verbose because of all the wrapping Condition in Field<Boolean> using DSL.field(Condition).
Alternatively, use e.g. NUM_NONNULLS() (Credits to Vik Fearing for this appraoch):
System.out.println(ctx.select(
field("num_nonnulls(variadic {0})", INTEGER, val(new Long { null }))
).fetch());
I have this string (character varying) as a row value in Postgresql :
{'img_0': 'https://random.com/xxxxxx.jpg', 'img_1': 'https://random.com/yyyyyy.jpg', 'img_2': 'https://random.com/zzzzzz.jpg'}
I am trying to json_eact_text() it but can't figure out how to.
I have tried to_jsonb() on it (working) and then jsonb_each(), but I have this error :
ERROR: cannot call jsonb_each on a non-object
My query :
WITH
test AS (
SELECT to_jsonb(value) as value FROM attribute_value WHERE id = 43918
)
SELECT jsonb_each(value) FROM test
Your text value is not valid JSON. JSON requires doublequotes (") to delimit strings.
This will work by doctoring your text provided that your data is consistently wrong:
with t (sometext) as (
values ($${'img_0': 'https://random.com/xxxxxx.jpg', 'img_1': 'https://random.com/yyyyyy.jpg', 'img_2': 'https://random.com/zzzzzz.jpg'}$$)
)
select jsonb_each_text(replace(sometext, '''', '"')::jsonb)
from t;
jsonb_each_text
---------------------------------------
(img_0,https://random.com/xxxxxx.jpg)
(img_1,https://random.com/yyyyyy.jpg)
(img_2,https://random.com/zzzzzz.jpg)
(3 rows)
To break this out into columns:
with t (sometext) as (
values ($${'img_0': 'https://random.com/xxxxxx.jpg', 'img_1': 'https://random.com/yyyyyy.jpg', 'img_2': 'https://random.com/zzzzzz.jpg'}$$)
)
select j.*
from t
cross join lateral jsonb_each_text(replace(sometext, '''', '"')::jsonb) as j;
key | value
-------+-------------------------------
img_0 | https://random.com/xxxxxx.jpg
img_1 | https://random.com/yyyyyy.jpg
img_2 | https://random.com/zzzzzz.jpg
(3 rows)
Folks, I'm trying to extract value of 'status' from below string(column name: people) in hive. The problem is, the column is neither a complete JSON nor stored as an Array.
I tried to make it look like a JSON by replacing '=' with ':', which didnt help.
[{name=abc, org=true, self=true, status=accepted, email=abc#gmail.com}, {name=cab abc, org=false, self=false, status=needsAction, email=cab#google.com}]
Below is the query I used:
SELECT
str.name,
str.org,
str.status
FROM table
LATERAL VIEW EXPLODE (TRANSLATE(people,'=',':')) exploded as str;
but I'm getting below error:
FAILED: UDFArgumentException explode() takes an array or a map as a parameter
Need output something like this:
name | org | status
-------- ------- ------------
abc | true | accepted
cab abc | false | needsAction
Note: There is a table already, the datatype is string, and I
can't change the table schema.
Solution for Hive. It possibly can be optimized. Read comments in the code:
with your_table as ( --your data example, you select from your table instead
select "[{name=abc, org=true, self=true, status=accepted, email=abc#gmail.com}, {name=cab abc, org=false, self=false, status=needsAction, email=cab#google.com}]" str
)
select --get map values
m['org'] as org ,
m['name'] as name ,
m['self'] as self ,
m['status'] as status ,
m['email'] as email
from
(--remove spaces after commas, convert to map
select str_to_map(regexp_replace(a.s,', +',','),',','=') m --map
from your_table t --replace w your table
lateral view explode(split(regexp_replace(str,'\\[|\\{|]',''),'}, *')) a as s --remove extra characters: '[' or '{' or ']', split and explode
)s;
Result:
OK
true abc true accepted abc#gmail.com
false cab abc false needsAction cab#google.com
Time taken: 1.001 seconds, Fetched: 2 row(s)