I'm using Oracle query with regexp_substr to extract json fields from JSON string. I would like just number(1177) of "pickupLocation": "1177" but I'm using this query didn't work
select to_char(regexp_substr(a.input_msg, '(\w*)("pickupLocation":")(\w*)(")',1,1))
from td_interface_phxsale_log a
output from my query : "pickupLocation":"1177"
myjson data :
{
"shiptoAddr": "",
"shippingCostAmt": "",
"pickupLocation": "1177"
}
Why using regular expressions? That makes absolutely no sense. You pay a lot of money for your Oracle license, use the JSON tools included with it.
with
inputs (my_json_input) as (
select to_clob('{
"shiptoAddr": "",
"shippingCostAmt": "",
"pickupLocation": "1177"
}')
from dual
)
-- End of simulated inputs (for testing only;
-- remove the WITH clause and use your actual table and column names below)
--
select json_value(my_json_input, '$.pickupLocation') as pickuplocation
from inputs
;
PICKUPLOCATION
---------------
1177
You don't need to_char() here; even when the JSON string is CLOB, json_value() returns varchar2 (unless explicitly requested otherwise with the returning clause).
If in fact the "pickup location" data type is number (as apparently it should be), you can stick returning number at the end of the json_value() call (before the closing parenthesis).
In general, you should not attempt to parse JSON content using regular expressions. Assuming you really needed to go down this path, you could try:
SELECT regexp_substr(input_msg, '"pickupLocation": "([^"]+)"', 1, 1, NULL, 1)
FROM td_interface_phxsale_log;
Demo
Note: Since JSON is already text, there is no need to convert what you extract using TO_CHAR.
Related
I am working on data in postgresql as in the following mytable with the fields id (type int) and val (type json):
id
val
1
"null"
2
"0"
3
"2"
The values in the json column val are simple JSON values, i.e. just strings with surrounding quotes and have no key.
I have looked at the SO post How to convert postgres json to integer and attempted something like the solution presented there
SELECT (mytable.val->>'key')::int FROM mytable;
but in my case, I do not have a key to address the field and leaving it empty does not work:
SELECT (mytable.val->>'')::int as val_int FROM mytable;
This returns NULL for all rows.
The best I have come up with is the following (casting to varchar first, trimming the quotes, filtering out the string "null" and then casting to int):
SELECT id, nullif(trim('"' from mytable.val::varchar), 'null')::int as val_int FROM mytable;
which works, but surely cannot be the best way to do it, right?
Here is a db<>fiddle with the example table and the statements above.
Found the way to do it:
You can access the content via the keypath (see e.g. this PostgreSQL JSON cheatsheet):
Using the # operator, you can access the json fields through the keypath. Specifying an empty keypath like this {} allows you to get your content without a key.
Using double angle brackets >> in the accessor will return the content without the quotes, so there is no need for the trim() function.
Overall, the statement
select id
, nullif(val#>>'{}', 'null')::int as val_int
from mytable
;
will return the contents of the former json column as int, respectvely NULL (in postgresql >= 9.4):
id
val_int
1
NULL
2
0
3
2
See updated db<>fiddle here.
--
Note: As pointed out by #Mike in his comment above, if the column format is jsonb, you can also use val->>0 to dereference scalars. However, if the format is json, the ->> operator will yield null as result. See this db<>fiddle.
I am trying to fix an array in a dataset. Currently, I have a data set that has a reference number to multiple different uuids. What I would like to do is flatten this out in Snowflake to make it so the reference number has separate row for each uuid. For example
Reference UUID
1) 9f823c2a-ced5-4dbe-be65-869311462f75 "[
""05554f65-6aa9-4dd1-6271-8ce2d60f10c4"",
""df662812-7f97-0b43-9d3e-12f64f504fbb"",
""08644a69-76ed-ce2d-afff-b236a22efa69"",
""f1162c2e-eeb5-83f6-5307-2ed644e6b9eb"",
]"
Should end up looking like:
Reference UUID
1) 9f823c2a-ced5-4dbe-be65-869311462f75 05554f65-6aa9-4dd1-6271-8ce2d60f10c4
2) 9f823c2a-ced5-4dbe-be65-869311462f75 df662812-7f97-0b43-9d3e-12f64f504fbb
3) 9f823c2a-ced5-4dbe-be65-869311462f75 08644a69-76ed-ce2d-afff-b236a22efa69
4) 9f823c2a-ced5-4dbe-be65-869311462f75 f1162c2e-eeb5-83f6-5307-2ed644e6b9eb
I just started working in Snowflake so I am new to it. It looks like there is a lateral flatten, but this is either not working on telling me that I have all sorts of errors with it. The documentation from snowflake is a bit perplexing when it comes to this.
While FLATTEN is the right approach when exploding an array, the UUID column value shown in the original description is invalid if interpreted as JSON syntax: "[""val1"", ""val2""]" and that'll need correction before a LATERAL FLATTEN approach can be applied by treating it as a VARIANT type.
If your data sample in the original description is a literal one and applies for all columnar values, then the following query will help transform it into a valid JSON syntax and then apply a lateral flatten to yield the desired result:
SELECT
T.REFERENCE,
X.VALUE AS UUID
FROM (
SELECT
REFERENCE,
-- Attempts to transform an invalid JSON array syntax such as "[""a"", ""b""]"
-- to valid JSON: ["a", "b"] by stripping away unnecessary quotes
PARSE_JSON(REPLACE(REPLACE(REPLACE(UUID, '""', '"'), '["', '['), ']"', ']')) AS UUID_ARR_CLEANED
FROM TABLENAME) T,
LATERAL FLATTEN(T.UUID_ARR_CLEANED) X
If your data is already in a valid VARIANT type with a successful PARSE_JSON done for the UUID column during ingest, and the example provided in the description was just a formatting issue that only displays the JSON invalid in the post, then the simpler version of the same query as above will suffice:
SELECT REFERENCE, X.VALUE AS UUID
FROM TABLENAME, LATERAL FLATTEN(TABLENAME.UUID) X
Is there a way to get a number formatted with a comma for thousand in numbers?
According to IBM documentation, this is the syntax:
DECIMAL(:newsalary, 9, 2, ',')
newsalary is the string (field)
9 is the precision
2 is the scale
, is the delimiter.
I tried:
SELECT DECIMAL ( T1.FIELD1 , 15 , 2 , "," ) AS TOTAL FROM TABLE T1
When trying it, I am getting the following error:
Message: [SQL0171] Argument 4 of function DECIMAL not valid.
DECIMAL converts from string type to a numeric type.
Numeric types don't have separators; only character representations of numbers have separators.
What tool are you using STRSQL, Run SQL Scripts or something else? Once you convert the string to a number, the tool should add the language appropriate separators when it displays the numeric data. For example, in STRSQL:
select decimal('12345.67', 12,2) as mynum
from sysibm.sysdummy1
Returns:
MYNUM
12,345.67
Using SQL to format strings is usually a bad idea. That should be left to whatever is consuming the data.
But if you really, really, really want to do it. You should create a user defined function (UDF) that does it for you. Here's an article, Make SQL Edit the Way You Want It To that includes source for for an EDITDEC function written in ILE RPG along with the SQL function definition you need to use it in an SQL statement.
I want to generate an XML element using Oracle's XML documentation generation support features that looks like this
<Example Attr=""></Example>
Attr is an attribute of element Example and has a value of empty string "".
When I tried to generate an XML Element using Oracle's XML functions, I couldn't generate an XML element which has an attribute whose value is an empty string.
select XMLELEMENT("hello", xmlattributes('' as "Max")) from dual
The result of the above query is
<hello></hello>
Note: there is no space between the single quotes for Max attribute.
However my requirement is
<hello Max=""></hello> -- there is no space between the double quotes.
Is there a way to do this?
As you're aware, for XMLAtttribute "if the value_expr is null, then no attribute is created for that value expression".
You can work around this with InsertChildXML but it isn't terribly pretty:
select insertchildxml(xmlelement("hello"), '/hello', '#Max', null) from dual;
INSERTCHILDXML(XMLELEMENT("HELLO"),'/HELLO','#MAX',NULL)
--------------------------------------------------------------------------------
<hello Max=""/>
... and as you can see it collapses an empty node, but that's only a potentially issue if you want this to look exactly as you showed - it's valid XML still. There is an even uglier way around that if you really need to.
That also suggests an alternative to #smnbbrv's replace:
select updatexml(xmlelement("hello", xmlattributes('$$IMPOSSIBLE-VALUE$$' as "Max")),
'/hello[#Max="$$IMPOSSIBLE-VALUE$$"]/#Max', null) from dual;
UPDATEXML(XMLELEMENT("HELLO",XMLATTRIBUTES('$$IMPOSSIBLE-VALUE$$'AS"MAX")),'/HEL
--------------------------------------------------------------------------------
<hello Max=""/>
which might be easier if your max attribute value is coming from data as you can NVL it to the impossible value. I'm not a fan of using magic values though really.
What about setting the property value to some impossible value and then replace it with the value you need (so, empty string in your case)?
select replace(
XMLELEMENT("hello", xmlattributes('$$IMPOSSIBLE-VALUE$$' as "Max")).getStringVal(),
'$$IMPOSSIBLE-VALUE$$'
)
from dual;
I assume you anyway in the end need the string value, so even if this XMLELEMENT is just an example of the problem and you have a biiiig XML generated, you still can generate it first and then, finally, replace all the values with one command as shown above.
I'm using this approach (you write the source attribute instead of the "null"). This way, you don't have to search for some overall impossible value, it's enough to be impossible for that attribute.
select
replace(XMLELEMENT("hello", xmlattributes(nvl(null,' ') as "Max")).getstringval(),
'Max=" "',
'Max=""')
from dual
I used updatexml() similar to A. Poole's example to create an empty string
set serveroutput on
DECLARE
xa xmltype;
xb xmltype;
BEGIN
xa := xmltype('<surfbreaks>'||
'<break lineNum="0" recordtype="empty">d street </break>'||
'<break lineNum="0" recordtype="empty">recordtype="empty" </break>'||
'</surfbreaks>');
dbms_output.put_line ('Before:');
dbms_output.put_line (xa.getclobval);
select UPDATEXML(xa, '//#recordtype', '') into xb from dual;
dbms_output.put_line ('After:');
dbms_output.put_line (xb.getclobval);
END; /
(Other than using a UDF) Any REGEXP-In-SQL support for DB2 9.7 ?
Starting with DB2 11.1 there is built-in regex support. One of the new function is REGEXP_SUBSTR and there are some more.
SELECT REGEXP_SUBSTR('hello to you', '.o',1,1)
FROM sysibm.sysdummy1
I'm komikoni(Keisuke Konishi).
I created the regular expression function (UDF) which does not exist in db2.
The UDF using the SQL/XML(Xquery).
You can easily install.
List of regular expressions provide UDF
REG_MATCHES
provides Coincidence existence ( Scalar )
REG_REPLACE
string substitution ( Scalar )
REG_COUNT
number of matches retrieved ( Scalar )
REG_POSITION
match position acquisition ( Scalar )
REG_SUBSTR
gets a string matching ( Scalar )
REG_SUBSTR_TABLE
list of matching string information ( Table )
REG_TOKENIZE_TABLE
list of mismatched string information (divided by a separator string) ( Table )
REG_ALLTOKEN_TABLE
list of mismatch string and matching string information ( Table )
Scripts can be downloaded from here.
(Sorry in Japanese)
https://www.ibm.com/developerworks/jp/data/library/db2/j_d-regularexpression/
(English : Machine translation Script : The last of a Japanese page)
I look forward to your feedback and comments.
The real answer is that DB2 does support regular expression since PureXML was added (v9.7 included) via xQuery with the matches function.
For example:
db2 "with val as (
select t.text
from texts t
where xmlcast(xmlquery('fn:matches(\$TEXT,''^[A-Za-z 0-9]*$'')') as integer) = 0
)
select * from val"
For more information:
http://publib.boulder.ibm.com/infocenter/db2luw/v9r7/topic/com.ibm.db2.luw.xml.doc/doc/xqrfnmat.html
That works fine except for DB2 z/OS - in DB2 v10 z/OS you must use PASSING as follows
with val as (
select t.text
from texts t
where xmlcast(xmlquery('fn:matches($v,"^[A-Za-z 0-9]*$")'
PASSING t.text as "v" ) as integer) = 0
)
select * from val
There is no built-in support for regular expressions in DB2 9.7.
The only way is using UDFs or table functions as described in the article 'OMG Ponies' added in the comment.
#dan1111: I do not appreciate my post being edited, especially if people can't read the question correctly. The OP asked Any REGEXP-In-SQL support for DB2 9.7
SQL is not XQuery !!!
Sorry, don't delete the text of my 100% correct answer. You can add a comment or write your own answer.