I have data in the comment column of the payments table. The data is stored as plain text in the following format:
{"foo"=>"bar"}
I need to query the value of the specific "foo" key and tried the following:
select comment::json -> 'foo' from payments
but because the data stored is not in JSON format I get the following error:
invalid input syntax for type json DETAIL: Token "=" is invalid. CONTEXT: JSON data, line 1: {"foo"=>"bar"}
which refers to the => that Ruby uses for Hashes.
Is there a way to convert the text data to JSON data on-the-fly so I can then access the specific keys I need?
You can replace the => with a : to make that example a valid JSON value:
replace(comment, '=>', ':')::jsonb ->> 'foo'
It sounds like the data is technically valid ruby which means we can do something a bit clever.
require 'json'
def parse_data(data_string)
eval(data_string).to_json
end
Should do the trick so long as the data is trusted.
Related
I'm having issues with parsing a json array and loading into a hive table.
Json array lives in table dmetrics and in column metrics
The json looks like this"
{"impressions":8256,"requests":67.....
I keep getting the following error :
{"cannot resolve 'get_json_object(dmetrics.metrics, '$.imppressions')' due to data type mismatch: argument 1 requires string type, however, 'dmetrics.metrics' is of struct<confirmedimpressions:bigint,requests:bigint,....
I'm using the following code:
get_json_object(dmetrics.metrics, '$.impression.included)
I log messages that are JSON objects. The JSON has an array that contains key/value pairs:
{
...
"arr": [{"key": "foo", "value": "bar"}, ...],
...
}
Now I want to filter results that contains a specific key and extract the values for a specific key in the array.
I've tried using regex, something like parse #message /.*"key":"my_specific_key","value":(?<value>.*}).*/ which extracts the value but also returns the rest of the message. Also it doesn't filter the results.
How can I filter results and extract the values for a specific key?
If in your log entry in the cloudwatch log group they are actually showing up as json, you can just reference the key directly in any place you would a field.
(don't need the #, cloudwatch appends that automatically to all default values)
If you are using python, you can use aws_lambda_powertools to do this as well, in a very slick way (and its an actual aws product)
If they are showing up in your log as a string, then it may be an escaped string and you'll have to match it -exactly- - including spaces and what not. when you parse, you will want to do something like this:
if this is the string of your log message '{"AKey" : "AValue", "Key2" : "Value2"}
parse #message "{\"*\" : \"*\",\"*\" : \"*\"} akey, akey_value, key2, key2_value
then you can filter or count or anything against those variables. parse is specifically a statement to match a pattern and assign the wildcard to a variable, one at a time in order
tho with a complex json, if your above regex works than all you need is a filter statement
field #message
| pares #message ... your regex as value_var
| filer value_var /some more regex/
if its not a string in the log entry, but an actual json, you can just reference against the key:
filter a_key ~="some value" (or regex here)
https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_AnalyzeLogData-discoverable-fields.html
for more info
I have column response_payload in table response as below
{"studentName":"Karate","studentStream":"API Testing", "studentAge":18}
Now I need to get the new json object based on my keys. For example, if my keys are studentName and studentStream I must get
{"studentName":"Karate","studentStream":"API Testing"}
In case if I provide invalid key, it should not bring anything related to invalid key.
I have tried json_build_object built-in function in postgresql, it is bringing invalid key and value as below
select json_build_object('invalidKey', payload -> 'invalidValue')
from response
O/P:- {"invalidKey":"null"}
I want output when invalid key and value is provided, it should not bring anything as below
O/p:- {}
You can use jsonb_strip_nulls
select
jsonb_strip_nulls(jsonb_build_object('invalidKey', payload -> 'invalidValue'))
from response
I am having trouble working with the JSONB structure in PostgreSQL. currently my data is saved as follows:
"{\"Hello\":\"World\",\"idx\":0}"
Which obviously is not correct 😀 so I am trying to "repair" this and get the actual JSON representation for querying with:
SELECT regexp_replace(trim('"' FROM json_data::text), '\\"', '"', 'g')::jsonb FROM My_table
However when trying this, I get the following error:
ERROR: invalid input syntax for type json
DETAIL: Token "Рыба" is invalid.
CONTEXT: JSON data, line 1: ...х, как : Люди X, Пароль \\"Рыба...
SQL state: 22P02
So I am thinking that this is due to the character encoding that is not being accepted by the JSONB standard.
My main question then is though, how can I repair this kind of table so that I am still able to query it? I tried utilizing conver_from and convert_to but am unable to figure out how to fix this error... did anyone encounter this already?
Update: Found it! (thanks to Convert JSON string to JSONB), utilizing
SELECT (json_data#>>'{}')::jsonb FROM my_table
fixed it
Im am trying to save some special characters in a json column of a table. but this is actually not working
this is how it is stored
{"district":"\u099a\u09be\u0981\u09a6\u09aa\u09c1\u09b0",
"sub_district":"\u099a\u09be\u0981\u09a6\u09aa\u09c1\u09b0"}
i am storing my input values as below..i am using laravel
present = array('district'=>$request->district,'sub_district'=>$request->sub_districtce);
$card->present_address = json_encode($present);
and while searching for a string in that json object, i am using a query below
$allowances = Card::SELECT('id','name','nid','village')
->where(DB::raw("json_extract((present_address), '$.district')"), চাদপুর)
->get();
can anybody help me on this situation where i can store special characters/unicodes in that json object.?
It is perfectly normal. As soon as you will do json_decode(), you will get back your expected values:
array (
'district' => 'চাঁদপুর',
'sub_district' => 'চাঁদপুর',
)