I have the following sql code in oracle client 11g. I would like to convert the "ssno" to md5 hash. I've read other posts but none of them specifically say where to put the code. thanks!
SELECT FS_HIRES."rsa",
FS_HIRES."ssno",
FS_HIRES."lname",
FS_HIRES."series",
FS_HIRES."grade",
FS_HIRES."pos_title",
FS_HIRES."ethnicity",
FS_HIRES."disability",
FS_HIRES."type_appt",
FS_HIRES."Perm_Temp",
FS_HIRES."gender",
FS_HIRES."age",
FS_HIRES."age_categories",
FS_HIRES."los",
FS_HIRES."date_apnt",
FS_HIRES."mm_apnt",
FS_HIRES."yy_apnt",
FS_HIRES."apnt_noa",
FS_HIRES."apnt_auth",
FS_HIRES.L2_DESC,
FS_HIRES.L3_DESC,
FS_HIRES.L4_DESC,
FS_HIRES.L5_DESC,
FS_HIRES."fy"
FROM FS_HIRES
The HASH_MD5 constant can't be referred to directly from SQL, so a statement like:
SELECT FS_HIRES."rsa",
DBMS_CRYPTO.HASH(UTL_I18N.STRING_TO_RAW(FS_HIRES."ssno", 'AL32UTF8'),
DBMS_CRYPTO.HASH_MD5),
...
will get an error like "ORA-06553: PLS-221: 'HASH_MD5' is not a procedure or is undefined". You can either use the internal value for that constant, which is 2:
SELECT FS_HIRES."rsa",
DBMS_CRYPTO.HASH(UTL_I18N.STRING_TO_RAW(FS_HIRES."ssno", 'AL32UTF8'), 2),
...
Or if you don't want to rely on a constant that could potentially change in a future release, define your own function:
create or replace function my_md5(p_str varchar2) return raw is
begin
return dbms_crypto.hash(utl_i18n.string_to_raw(p_str, 'AL32UTF8'),
dbms_crypto.hash_md5);
end my_md5;
/
... and then call that:
SELECT FS_HIRES."rsa",
MY_MD5(FS_HIRES."ssno", 'AL32UTF8') AS "ssno",
...
If your database character isn't AL32UTF8, you may need to do more conversion as mentioned in the documentation, and it'll be easier to hide that in the function too.
Try this:
select
'123456789' as ssno,
rawtohex(
DBMS_CRYPTO.Hash (
UTL_I18N.STRING_TO_RAW ('123456789', 'AL32UTF8'),
2)
) as ssno_md5
from dual;
Output:
SSNO SSNO_MD5
123456789 25F9E794323B453885F5181F1B624D0B
Related
I've a query like this:
SELECT personas.IDDNI iddni,
PERSONAS.NOMBRE NOMBRE,
PERSONAS.APELLIDO1 APELLIDO1,
PERSONAS.APELLIDO2 APELLIDO2,
PERSONAS.ANTIGUEDAD ANTIGUEDAD,
VACACIONES(personas.IDDNI, to_char(add_months(sysdate, 0), 'YYYY')) VACACIONES
FROM personas personas,
trieniosobservaciones t
WHERE personas.iddni = t.iddni
AND ( personas.iddni = '47656567' )
I'd like to know what VACACIONES(personas.IDDNI,to_char(add_months(sysdate,0),'YYYY')) VACACIONES does in the query, as depending on the personas.iddni value it can return one row or give the following error:
The number specified in exact fetch is less than the rows returned.
VACACIONES is a user-defined function that takes two arguments, an IDDNI value for a person and a year. Beyond that, we cannot tell you what it does because it is a user-defined function and we do not have access to your database or the source code of the function.
You can find the source-code of the function using:
SELECT *
FROM all_source
WHERE name = 'VACACIONES'
AND type = 'FUNCTION'
ORDER BY owner, line;
and then you can work out what it does.
Oracle DB.
Spring JPA using Hibernate.
I am having difficulty inserting a Clob value into a native sql query.
The code calling the query is as follows:
#SuppressWarnings("unchecked")
public List<Object[]> findQueryColumnsByNativeQuery(String queryString, Map<String, Object> namedParameters)
{
List<Object[]> result = null;
final Query query = em.createNativeQuery(queryString);
if (namedParameters != null)
{
Set<String> keys = namedParameters.keySet();
for (String key : keys)
{
final Object value = namedParameters.get(key);
query.setParameter(key, value);
}
}
query.setHint(QueryHints.HINT_READONLY, Boolean.TRUE);
result = query.getResultList();
return result;
}
The query string is of the format
SELECT COUNT ( DISTINCT ( <column> ) ) FROM <Table> c where (exact ( <column> , (:clobValue), null ) = 1 )
where "(exact ( , (:clobValue), null ) = 1 )" is a function and "clobValue" is a Clob.
I can adjust the query to work as follows:
SELECT COUNT ( DISTINCT ( <column> ) ) FROM <Table> c where (exact ( <column> , to_clob((:stringValue)), null ) = 1 )
where "stringValue" is a String but obviously this only works up to the max sql string size (4000) and I need to pass in much more than that.
I have tried to pass the Clob value as a java.sql.Clob using the method
final Clob clobValue = org.hibernate.engine.jdbc.ClobProxy.generateProxy(stringValue);
This results in a java.io.NotSerializableException: org.hibernate.engine.jdbc.ClobProxy
I have tried to Serialize the Clob using
final Clob clob = org.hibernate.engine.jdbc.ClobProxy.generateProxy(stringValue);
final Clob clobValue = SerializableClobProxy.generateProxy(clob);
But this appears to provide the wrong type of argument to the "exact" function resulting in (org.hibernate.engine.jdbc.spi.SqlExceptionHelper:144) - SQL Error: 29900, SQLState: 99999
(org.hibernate.engine.jdbc.spi.SqlExceptionHelper:146) - ORA-29900: operator binding does not exist
ORA-06553: PLS-306: wrong number or types of arguments in call to 'EXACT'
After reading some post about using Clobs with entities I have tried passing in a byte[] but this also provides the wrong argument type (org.hibernate.engine.jdbc.spi.SqlExceptionHelper:144) - SQL Error: 29900, SQLState: 99999
(org.hibernate.engine.jdbc.spi.SqlExceptionHelper:146) - ORA-29900: operator binding does not exist
ORA-06553: PLS-306: wrong number or types of arguments in call to 'EXACT'
I can also just pass in the value as a String as long as it doesn't break the max string value
I have seen a post (Using function in where clause with clob parameter) which seems to suggest that the only way is to use "plain old JDBC". This is not an option.
I am up against a hard deadline so any help is very welcome.
I'm afraid your assumptions about CLOBs in Oracle are wrong. In Oracle CLOB locator is something like a file handle. And such handle can be created by the database only. So you can not simply pass CLOB as bind variable. CLOB must be somehow related to database storage, because this it can occupy up to 176TB and something like that can not be held in Java Heap.
So the usual approach is to call either DB functions empty_clob() or dbms_lob.create_temporary (in some form). Then you get a clob from database even if you think it is "IN" parameter. Then you can write as many data as you want into that locator (handle, CLOB) and then you can use this CLOB as a parameter for a query.
If you do not follow this pattern, your code will not work. It does not matter whether you use JPA, SpringBatch or plan JDBC. This constrain is given by the database.
It seems that it's required to set type of parameter explicitly for Hibernate in such cases. The following code worked for me:
Clob clob = entityManager
.unwrap(Session.class)
.getLobHelper()
.createClob(reader, length);
int inserted = entityManager
.unwrap(org.hibernate.Session.class)
.createSQLQuery("INSERT INTO EXAMPLE ( UUID, TYPE, DATA) VALUES (:uuid, :type, :data)")
.setParameter("uuid", java.util.Uuid.randomUUID(), org.hibernate.type.UUIDBinaryType.INSTANCE)
.setParameter("type", java.util.Uuid.randomUUID(), org.hibernate.type.StringType.INSTANCE)
.setParameter("data", clob, org.hibernate.type.ClobType.INSTANCE)
.executeUpdate();
Similar workaround is available for Blob.
THE ANSWER: Thank you both for your answers. I should have updated this when i solved the issue some time ago. In the end I used JDBC and the problem disappeared in a puff of smoke!
I want call function in case when but when i call function I get an error .
My code is :
s = case
when v.IsAverage=1 then
isnull(avg([Value]),0)
when v.IsCumulative=1 then
isnull(sum([Value]),0)
when v.IsCumulative=0 then
GetLastValueTest('93/.1/01','93/12/29')
I get this error :
'GetLastValueTest' is not a recognized built-in function name.
but when I call this way it works :
select * from GetLastValueTest('93/.1/01','93/12/29')
select * from GetLastValueTest('93/.1/01','93/12/29')
GetLastValueTest looks like a table-valued function
then correct way to use it in case statement is:
case
when v.IsAverage=1 then isnull(avg([Value]),0)
when v.IsCumulative=1 then isnull(sum([Value]),0)
when v.IsCumulative=0 then (select top 1 [ColumnName] from GetLastValueTest('93/.1/01','93/12/29'))
end
Assuming your GetLastValueTest is scalar function - you need to specify schema explicitly, i.e. dbo.GetLastValueTest('93/.1/01','93/12/29') (or whatever you schema is if it different from dbo)
If your GetLastValueTest is table-valued function then you can't mix select from it with scalar values in other branches of case. In this case you need something like
when v.IsCumulative=0
then (select top 1 some_value from dbo.GetLastValueTest('93/.1/01','93/12/29'))
This is my first db trigger. It compiles with warnings and therefore doesn't work. I've re-read the Oracle docs and searched online but can't work out where I'm going wrong. Any help with my trigger below would gratefully received.
CREATE OR REPLACE TRIGGER oa_mhd_update AFTER
INSERT ON men_mhd FOR EACH row WHEN (new.mhd_tktc LIKE 'OA_A_%'
OR new.mhd_tktc LIKE 'OA_T_%'
OR new.mhd_tktc LIKE 'OA_M_%')
DECLARE seq_var NVARCHAR2 (20);
BEGIN
SELECT (MAX (seq) + 1) into seq_var FROM oa_mhd_data;
INSERT
INTO oa_mhd_data
(
mhd_code,
seq,
mhd_mst1,
mhd_mst2,
mhd_cred,
mhd_cret,
mhd_tsks,
mhd_msgs,
mhd_tktc,
mhd_tref,
mhd_actn,
mhd_eref,
mhd_subj,
mhd_udf1,
mhd_udf2,
mhd_udf3,
mhd_udf4,
mhd_udf5,
mhd_udf6,
mhd_udf7,
mhd_udf8,
mhd_udf9,
mhd_udfa,
mhd_udfb,
mhd_udfc,
mhd_udfd,
mhd_udfe,
mhd_udff,
mhd_udfg,
mhd_udfh,
mhd_udfi,
mhd_udfj,
mhd_udfk,
mhd_updd,
mhd_begd,
mhd_begt,
mhd_endd,
mhd_endt,
mhd_mrcc,
mhd_mhdc,
mhd_mscc,
mhd_pprc,
mhd_ppss,
mhd_inst
)
VALUES
(
:new.mhd_code
seq_var,
:new.mhd_mst1,
:new.mhd_mst2,
:new.mhd_cred,
:new.mhd_cret,
:new.mhd_tsks,
:new.mhd_msgs,
:new.mhd_tktc,
:new.mhd_tref,
:new.mhd_actn,
:new.mhd_eref,
:new.mhd_subj,
:new.mhd_udf1,
:new.mhd_udf2,
:new.mhd_udf3,
:new.mhd_udf4,
:new.mhd_udf5,
:new.mhd_udf6,
:new.mhd_udf7,
:new.mhd_udf8,
:new.mhd_udf9,
:new.mhd_udfa,
:new.mhd_udfb,
:new.mhd_udfc,
:new.mhd_udfd,
:new.mhd_udfe,
:new.mhd_udff,
:new.mhd_udfg,
:new.mhd_udfh,
:new.mhd_udfi,
:new.mhd_udfj,
:new.mhd_udfk,
:new.mhd_updd,
:new.mhd_begd,
:new.mhd_begt,
:new.mhd_endd,
:new.mhd_endt,
:new.mhd_mrcc,
:new.mhd_mhdc,
:new.mhd_mscc,
:new.mhd_pprc,
:new.mhd_ppss,
:new.mhd_inst
)
END;
/
You're missing a comma between the first two elements of the values clause, and a semi-colon at the end of the insert statement:
VALUES
(
:new.mhd_code
seq_var,
:new.mhd_mst1,
...
:new.mhd_ppss,
:new.mhd_inst
)
... should be:
VALUES
(
:new.mhd_code,
seq_var,
:new.mhd_mst1,
...
:new.mhd_ppss,
:new.mhd_inst
);
Odd that you can't see the error though.
Incidentally, the max(seq) + 1 from ... pattern isn't reliable in a multi-user environment. It would be more normal (and safer) to use a proper sequence to generate that value.
Hi there are two syntactical errors
First please add a comma between two values you are inserting
VALUES
(
:new.mhd_,
seq_var,
:new.mhd_mst1,...
and second please add a semi colon at he end of insert statement
...
:new.mhd_pprc,
:new.mhd_ppss,
:new.mhd_inst
);
Hope this will solve your problem
Is there an easy way to do URL decoding within the BigQuery query language? I'm working with a table that has a column containing URL-encoded strings in some values. For example:
http://xyz.com/example.php?url=http%3A%2F%2Fwww.example.com%2Fhello%3Fv%3D12345&foo=bar&abc=xyz
I extract the "url" parameter like so:
SELECT REGEXP_EXTRACT(column_name, "url=([^&]+)") as url
from [mydataset.mytable]
which gives me:
http%3A%2F%2Fwww.example.com%2Fhello%3Fv%3D12345
What I would like to do is something like:
SELECT URL_DECODE(REGEXP_EXTRACT(column_name, "url=([^&]+)")) as url
from [mydataset.mytable]
thereby returning:
http://www.example.com/hello?v=12345
I would like to avoid using multiple REGEXP_REPLACE() statements (replacing %20, %3A, etc...) if possible.
Ideas?
Below is built on top of #sigpwned answer, but slightly refactored and wrapped with SQL UDF (which has no limitation that JS UDF has so safe to use)
#standardSQL
CREATE TEMP FUNCTION URLDECODE(url STRING) AS ((
SELECT SAFE_CONVERT_BYTES_TO_STRING(
ARRAY_TO_STRING(ARRAY_AGG(
IF(STARTS_WITH(y, '%'), FROM_HEX(SUBSTR(y, 2)), CAST(y AS BYTES)) ORDER BY i
), b''))
FROM UNNEST(REGEXP_EXTRACT_ALL(url, r"%[0-9a-fA-F]{2}|[^%]+")) AS y WITH OFFSET AS i
));
SELECT
column_name,
URLDECODE(REGEXP_EXTRACT(column_name, "url=([^&]+)")) AS url
FROM `project.dataset.table`
can be tested with example from question as below
#standardSQL
CREATE TEMP FUNCTION URLDECODE(url STRING) AS ((
SELECT SAFE_CONVERT_BYTES_TO_STRING(
ARRAY_TO_STRING(ARRAY_AGG(
IF(STARTS_WITH(y, '%'), FROM_HEX(SUBSTR(y, 2)), CAST(y AS BYTES)) ORDER BY i
), b''))
FROM UNNEST(REGEXP_EXTRACT_ALL(url, r"%[0-9a-fA-F]{2}|[^%]+")) AS y WITH OFFSET AS i
));
WITH `project.dataset.table` AS (
SELECT 'http://example.com/example.php?url=http%3A%2F%2Fwww.example.com%2Fhello%3Fv%3D12345&foo=bar&abc=xyz' column_name
)
SELECT
URLDECODE(REGEXP_EXTRACT(column_name, "url=([^&]+)")) AS url,
column_name
FROM `project.dataset.table`
with result
Row url column_name
1 http://www.example.com/hello?v=12345 http://example.com/example.php?url=http%3A%2F%2Fwww.example.com%2Fhello%3Fv%3D12345&foo=bar&abc=xyz
Update with further quite optimized SQL UDF
CREATE TEMP FUNCTION URLDECODE(url STRING) AS ((
SELECT STRING_AGG(
IF(REGEXP_CONTAINS(y, r'^%[0-9a-fA-F]{2}'),
SAFE_CONVERT_BYTES_TO_STRING(FROM_HEX(REPLACE(y, '%', ''))), y), ''
ORDER BY i
)
FROM UNNEST(REGEXP_EXTRACT_ALL(url, r"%[0-9a-fA-F]{2}(?:%[0-9a-fA-F]{2})*|[^%]+")) y
WITH OFFSET AS i
));
It's a good feature request, but currently there is no built in BigQuery function that provides URL decoding.
One more workaround is using a user-defined function.
#standardSQL
CREATE TEMPORARY FUNCTION URL_DECODE(enc STRING)
RETURNS STRING
LANGUAGE js AS """
try {
return decodeURI(enc);;
} catch (e) { return null }
return null;
""";
SELECT ven_session,
URL_DECODE(REGEXP_EXTRACT(para,r'&kw=(\w|[^&]*)')) AS q
FROM raas_system.weblog_20170327
WHERE para like '%&kw=%'
LIMIT 10
I agree with everyone here that URLDECODE should be a native function. However, until that happens, it is possible to write a "native" URLDECODE:
SELECT id, SAFE_CONVERT_BYTES_TO_STRING(ARRAY_TO_STRING(ps, b'')) FROM (SELECT
id,
ARRAY_AGG(CASE
WHEN REGEXP_CONTAINS(y, r"^%") THEN FROM_HEX(SUBSTR(y, 2))
ELSE CAST(y AS bytes)
END ORDER BY i) AS ps
FROM (SELECT x AS id, REGEXP_EXTRACT_ALL(x, r"%[0-9a-fA-F]{2}|[^%]+") AS element FROM UNNEST(ARRAY['domodossola%e2%80%93locarno railway', 'gabu%c5%82t%c3%b3w']) AS x) AS x
CROSS JOIN UNNEST(x.element) AS y WITH OFFSET AS i GROUP BY id);
In this example, I've tried and tested the implementation with a couple of percent-encoded page names from Wikipedia as the input. It should work with your input, too.
Obviously, this is extremely unwieldly! For that reason, I'd suggest building a materialized join table, or wrapping this in a view, rather than using this expression "naked" in your query. However, it does appear to get the job done, and it doesn't hit the UDF limits.
EDIT: #MikhailBerylyant's post below has wrapped this cumbersome implementation into a nice, tidy little SQL UDF. That's a much better way to handle this!