I've written a plpgsql script which generates an array of json objects in a string but after I use to_json() method passing a variable with that string to it, it returns a result which is doublequoted and also every doublequote character is escaped. But I need that string as is.
initial content of jsonResult variable is:
[{"key":{04949429911,"Code":"400"},"value":"20000.00"},{"key":{"InsuranceNumber":"04949429911","Code":"403"},"value":"10000.00"},...]
but after to_json() it looks like this:
"[{\"key\":{04949429911,\"Code\":\"400\"},\"value\":\"20000.00\"},{\"key\":{\"InsuranceNumber\":\"04949429911\",\"Code\":\"403\"},\"value\":\"10000.00\"}...]"
This is the place where everything stored in jsonResult breakes:
UPDATE factor_value SET params = to_json(jsonResult) WHERE id = _id;
What am I doing wrong?
This answer points out that simply casting to json should suffice:
UPDATE factor_value SET params = jsonResult::json WHERE id = _id;
The weird escaping you see is probably due to postgresql not realizing you already have valid JSON and your varchar is just converted to a plain javascript/JSON string.
I needed to convert a bytea column to jsonb and I ended up with escaped characters when using to_json. I also used bytea_data_column::jsonb and postgres said cannot cast to jsonb. Here is how I worked around it:
bytea_data_column::text::jsonb
Related
I have, in a database, records that are serialized PHP strings that I must obfuscate emails if there are any. The simplest record is like {s:20:"pika.chu#pokemon.com"}. It is basically saying: this is a string of length 20 which is pika.chu#pokemon.com. This field can be kilobytes long with lot of emails (or none) and sometimes it is empty.
I wish I could use a SQL regular expression function to obfuscate the user part of the email while preserving the length of the string in order not to break the PHP serialization. The example email above shall be turned into {s:20:"xxxxxxxx#pokemon.com"} where the number of x matches the length of pika.chu.
Any thoughts?
Here is a more complete example of what can be found as serialized PHP:
a:4:{s:7:"locales";a:3:{i:0;s:5:"fr_FR";i:1;s:5:"de_DE";i:2;s:5:"en_US";}s:9:"publisher";s:18:"john#something.com";s:7:"authors";a:2:{i:0;s:21:"william#something.com";i:1;s:19:"debbie#software.org";}s:12:"published_at";O:8:"DateTime":3:{s:4:"date";s:26:"2022-01-26 13:05:26.531289";s:13:"timezone_type";i:3;s:8:"timezone";s:3:"UTC";}}
I tried to do it using native functions but it not worked because functions like REGEXP_REPLACE don't let you manipulate the match to get the size of it, for example.
Instead, I've created a UDF to do that:
CREATE TEMP FUNCTION hideEmail(str STRING)
RETURNS STRING
LANGUAGE js AS """
return str
.replace(/([a-zA-Z.0-9_\\+-:]*)#/g, function(txt){return '*'.repeat(txt.length-1)+"#";})
""";
select hideEmail('a:4:{s:7:"locales";a:3:{i:0;s:5:"fr_FR";i:1;s:5:"de_DE";i:2;s:5:"en_US";}s:9:"publisher";s:18:"john#something.com";s:7:"authors";a:2:{i:0;s:21:"william#something.com";i:1;s:19:"debbie#software.org";}s:12:"published_at";O:8:"DateTime":3:{s:4:"date";s:26:"2022-01-26 13:05:26.531289";s:13:"timezone_type";i:3;s:8:"timezone";s:3:"UTC";}}')
Result:
a:4:{s:7:"locales";a:3:{i:0;s:5:"fr_FR";i:1;s:5:"de_DE";i:2;s:5:"en_US";}s:9:"publisher";s:18:"****#something.com";s:7:"authors";a:2:{i:0;s:21:"*******#something.com";i:1;s:19:"******#software.org";}s:12:"published_at";O:8:"DateTime":3:{s:4:"date";s:26:"2022-01-26 13:05:26.531289";s:13:"timezone_type";i:3;s:8:"timezone";s:3:"UTC";}}
I am using Postgres DB and I have a JSONB column. I am trying to insert a json node, which contains a regex , just for storage purposes.
here is the regex string (email validator):
^[A-Z0-9_!#$%&'*+/=?`{|}~^-]+(?:\.[A-Z0-9_!#$%&'*+/=?`{|}~^-]+↵)*#[A-Z0-9-]+(?:\.[A-Z0-9-]+)*$
I am trying to insert this like so
{
"title": "Testing",
"myregex": "^[A-Z0-9_!#$%&'*+/=?`{|}~^-]+(?:\.[A-Z0-9_!#$%&'*+/=?`{|}~^-]+↵)*#[A-Z0-9-]+(?:\.[A-
Z0-9-]+)*$"
}
However, it keeps throwing error. How can I escape or otherwise get this regex string stored in jsonb?
I don't know that the string you are asking us to escape for your has already been escaped properly for this medium, so what I see as the string might not be what you want. When I copied it from my screen, I got in the middle of it some weird carriage-return like thing or something else non-ASCII, which I removed.
Assuming you got the string into PostgreSQL accurately in the first place as text (for which I use dollar quoting), then the to_jsonb function will convert it into properly escaped JSON
select to_jsonb($JJ$^[A-Z0-9_!#$%&'*+/=?`{|}~^-]+(?:\.[A-Z0-9_!#$%&'*+/=?`{|}~^-]+)*#[A-Z0-9-]+(?:\.[A-Z0-9-]+)*$$JJ$::text);
to_jsonb
---------------------------------------------------------------------------------------------------
"^[A-Z0-9_!#$%&'*+/=?`{|}~^-]+(?:\\.[A-Z0-9_!#$%&'*+/=?`{|}~^-]+)*#[A-Z0-9-]+(?:\\.[A-Z0-9-]+)*$"
Now I don't know if what you see is what I intended you to see, because I might need to escape the escaping of the escapement, which might have been de-escaped incorrectly in the first place. But the general principle should still apply.
Looking in your keywords, seems to be that you want to insert directly via pgadmin. So, PostgreSQL have native JSON format support, I did an example in how to use this:
To show how you can store json objects, follows a dummy table:
$ CREATE TABLE test (id INT, data JSON, datab JSONB);
CREATE TABLE
Trying to check your string on the fly, on basic way. I'm saying to PostgreSQL to convert the string in json and access the 'title' property:
test=# SELECT '{"title": "Testing"}'::json->'title';
?column?
-----------
"Testing"
(1 row)
Now, trying with your json (with some adjustments) - The backslash \ and quote ' are escaped here:
test=# SELECT '{"title": "Testing","myregex": "^[A-Z0-9_!#$%&''*+/=?`{|}~^-]+(?:\\.[A-Z0-9_!#$%&''*+/=?`{|}~^-]+↵)*#[A-Z0-9-]+(?:\\.[A-Z0-9-]+)*$"}'::json->'myregex'
?column?
----------------------------------------------------------------------------------------------------
"^[A-Z0-9_!#$%&'*+/=?`{|}~^-]+(?:\\.[A-Z0-9_!#$%&'*+/=?`{|}~^-]+↵)*#[A-Z0-9-]+(?:\\.[A-Z0-9-]+)*$"
(1 row)
So, to save, simply do the json conversion with escape strings:
test=# INSERT INTO test VALUES (1, '{"title": "Testing","myregex": "^[A-Z0-9_!#$%&''*+/=?`{|}~^-]+(?:\\.[A-Z0-9_!#$%&''*+/=?`{|}~^-]+↵)*#[A-Z0-9-]+(?:\\.[A-Z0-9-]+)*$"}'::json);
INSERT 1
And, we're done.
I need to execute a SQL query, which converts a String column to a Array and then validate the size of that array
I was able to do it easily with postgresql:
e.g.
select
cardinality(string_to_array('a$b','$')),
cardinality(string_to_array('a$b$','$')),
cardinality(string_to_array('a$b$$$$$','$')),
But for some reason trying to convert String on vertica to array is not that simple, Saw this links:
https://www.vertica.com/blog/vertica-quick-tip-dynamically-split-string/
https://forum.vertica.com/discussion/239031/how-to-create-an-array-in-vertica
And much more that non of them helped.
I also tried using:
select REGEXP_COUNT('a$b$$$$$','$')
But i get an incorrect value - 1.
How can i Convert String to array on Vertica and gets his Length ?
$ has a special meaning in a regular expression. It represents the end of the string.
Try escaping it:
select REGEXP_COUNT('a$b$$$$$', '[$]')
You could create a UDx scalar function (UDSF) in Java, C++, R or Python. The input would be a string and the output would be an integer. https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/ExtendingVertica/UDx/ScalarFunctions/ScalarFunctions.htm
This will allow you to use language specific array logic on the strings passed in. For example in python, you could include this logic:
input_list = input.split("$")
filtered_input_list = list(filter(None, input_list))
list_count = len(filtered_input_list)
These examples are a good starting point for writing UDx's for Vertica. https://github.com/vertica/UDx-Examples
I wasn't able to convert to an array - but Im able to get the length of the values
What i do is convert to Rows an use count - its not best performance wise
But with this way Im able to do also manipulation like filtering of each value between delimiter - and i dont need to use [] for characters like $
select (select count(1)
from (select StringTokenizerDelim('a$b$c','$') over ()) t)
Return 3
Is there a simple way (like convert or cast) to return a binary field as a string without actually converting it or casting it?
The data I want looks like this in the database:
0x24FC40460F58D64394A06684E9749F30
When using convert(varchar(max), binary_field), it comes back like this:
$ü#FXÖC” f„étŸ0
cast(binary_field as varchar(max)) had the same results.
I'm trying to modify a datasource in an old vb.net application, so I want it to be a string like the existing values, but still look like the original data. I don't have to do it in the query, but I'd like to if possible to avoid modifying the old vb code too much.
Here is my select statement:
SELECT strName, strDesc, binUUID
FROM MyTableName
where intFamily = '1234'
order by strName
Alternatively, if I do cast/convert it, would it be safe to compare "$ü#FXÖC” f„étŸ0" stored as a string at a later time to another value gathered in the same way?
There is a built in function to generate hex strings from binary values
SELECT sys.fn_varbintohexstr (0x24FC40460F58D64394A06684E9749F30)
I have a table in oracle db with data type RAW.
I would like export into an xml file but when I retrieve the data from the RAW column I get a [#4r5... instead of the value in db (123454678...)
How can I write this value in xml (the goal is to export in another db)
Thanks a lot.
You don't say what code you're using to access the column, but it looks like you are getting back a byte[]
You should be able to print the actual bytes in groovy using:
println obj.toList().join(',')
But if you were expecting a string or something, you'll need to know how it was converted to bytes in the first place...
If you want the byte array converted to an array of integers, you can just do:
List byteList = a.toList()
(or if you want your bytes to be treated as unsigned, you can do)
List byteList = a.collect { it & 0xff }
It depends what you intend to do with the array though... It is probably best left as an array...
Did you try PL/SQL?
declare
a mytable.rawcol%TYPE;
begin
select rawcol into a from mytable;
-- now do something with "a"
end;