I'm trying to understand the meaning of this regular expression function and it purpose in the select statement.
create or replace FUNCTION REPS_MTCH(string_orig IN VARCHAR2 , string_new IN VARCHAR2, score IN NUMBER)
RETURN PLS_INTEGER AS
BEGIN
IF string_orig IS NULL AND string_new IS NULL THEN
RETURN 0;
ELSIF utl_match.jaro_winkler_similarity(replace(REGEXP_REPLACE(UPPER(string_orig), '[^a-z|A-Z|0-9]+', ''),' ',''),replace(REGEXP_REPLACE(UPPER(string_new), '[^a-z|A-Z|0-9]+', ''),' ','')) >= score THEN
RETURN 1;
ELSE
RETURN 0;
END IF;
//the REPS_MTCH function is being called in this select statement. the select statement is to match names in the the Temp table name as REPS_MTCH_D_STDNT_TMP against the master table named as REPS_MTCH_D_STDNT_MSTR. what is the purpose of the REPS_MTCH function in this select statement?
SELECT
REPS_MTCH(REPS_MTCH_D_STDNT_TMP.FIRST_NAME,REPS_MTCH_D_STDNT_MSTR.FIRST_NAME,85) AS first_match_score,
what is the purpose of the REPS_MTCH function in this select statement?
In the above function the REGEXP_REPLACE is removing all occurrences any non alpha numeric or pipe (|) characters. After that the REGEXP_REPLACE is also wrapped in a redundant call to the regular REPLACE function which simply removes the spaces which were already removed by the REGEXP_REPLACE calls. The test could be rewritten as follows and still behave the identically since the inputs are first UPPERcased before the replace operations occur:
ELSIF utl_match.jaro_winkler_similarity(
REGEXP_REPLACE(UPPER(string_orig), '[^A-Z|0-9]+', '')
,REGEXP_REPLACE(UPPER(string_new) , '[^A-Z|0-9]+', '')
) >= score
THEN RETURN 1;
I simply removed the extra replace operation, the unnecessary lower case a-z and the extra pipe (|) character from the regular expression's character classes.
The JARO_WINKLER_SIMILARITY function just computes a score from 0 not similar to 100 identical of the remaining alpha numeric and pipe characters. You can check out the wikipedia entry on Jaro Winkler distances if you want to know more about them.
Related
I'm having : delimited column like 1:2:3:. I want to get this into 1,2,3. My query looks like,
select name
from status where id IN (SELECT REPLACE(NEXT_LIST,':',',')
FROM status);
but I got an error
ORA-01722: invalid number
(1, 2, 3, 4) is different from ('1, 2, 3, 4'). IN requires the former, a list of values; you give it the latter, a string.
You have two options mainly:
Build the query dynamically, i.e. get the list first, then use this to build a query string.
Tokenize the string. This can be done with a custom pipelined function or a recursive query, maybe also via some XML functions. Google "Oracle tokenize string" to find a method that suits you.
UPDATE Option #3: Use LIKE as in ':1:2:3:4:' like '%:3:%'
(This requires your next_list to contain only simple numbers separated with colons. No leading zeros, no blanks, no other characters.)
select name
from status
where (select ':' || next_list || ':' from status) like '%:' || id || ':%'
i agreed with Thorsten but i wonder if we just replace one more time would it works? i mean like this:
select name
from status where id IN (SELECT replace(REPLACE(NEXT_LIST,':',','),'''','')
FROM status);
The REPLACE function returns a string, so the nested query returns a list of string values (where colons replaced with commas), but not a list of number values. When Oracle engine interprets id IN (str_value) it tries to cast the str_value to number and raises exception ORA-01722: invalid number because there are cases like '1:2:3' which are definetely unparseable.
The "pure sql" approach leads us to using custom function detecting if a number is in a colon-separated list:
-- you need Oracle 12c to use function in the WITH clause
-- on earlier versions just unwrap CASE statement and put it into query
WITH
FUNCTION in_list(p_id NUMBER, p_list VARCHAR2) RETURN NUMBER DETERMINISTIC IS
BEGIN
RETURN CASE WHEN
instr(':' || p_list || ':', ':' || p_id || ':') > 0
THEN 1 ELSE 0 END;
END;
SELECT *
FROM status
WHERE in_list(id, next_list) = 1;
Here I assume that values in the next_list column are strings containing numbers separated with colon without spaces. In common case you shall modify the function to match specific list formats.
Input data:
abcdef_fhj_viji.dvc
Expected output:
fhj_viji.dvc
The part to be trimmed is not constant.
Use the REPLACE method
Select REPLACE('abcdef_fhj_viji.dvc','abcde','')
If you want this query for your table :
Select REPLACE(column,'abcde','') from myTable
For update :
UPDATE TABLE
SET column = REPLACE(column,'abcde','')
select substr('abcdef_fhj_viji.dvc',instr('abcdef_fhj_viji.dvc','_')+1) from dual
So, Its all depends on INSTR function, define from which position and which occurrence, you will get the index and pass that index to SUBSTR to get your string.
Since you didn't give a lot of information I'm gonna assume some.
Let's assume you want a prefix of some string to be deleted. A good way to do that is by using Regular Expressions. There's a function called regexp_replace, that can find a substring of a string, depending on a pattern, and replace it with a different string. In PL/SQL you could write yourself a function using regexp_replace, like this:
function deletePrefix(stringName in varchar2) return varchar2 is
begin
return regexp_replace(stringName, '^[a-zA-Z]+_', '');
end;
or just use this in plain sql like:
regexp_replace(stringName, '^[a-zA-Z]+_', '');
stringName being the string you want to process, and the ^[a-zA-Z]+_ part depending on what characters the prefix includes. Here I only included upper- and lowercase letters.
I have a number of tables 'App_build', 'Server_build' with a column called 'buildid' and it contains a large number of records. I.e.:
buildid
-----------
Application1_BLD_01
Application1_BLD_02
Application1_BLD_03
Application2_BLD_01
Application3_BLD_01
Application3_BLD_02
Application4_1_0_0_1 - old format to be disregarded
Application4_1_0_0_2
Application4_BLD_03
I want to write a function called getmax(tablename) i.e. getmax('App_build')
which will return a recordset which lists the highest values only. I.e:
buildid
--------
Application1_BLD_03
Application2_BLD_01
Application3_BLD_02
Application4_BLD_03
I am new to SQL so am not sure how to start - I guess I can use a split command and then the MAX function but I have no idea where to start.
Any help will be great.
Assuming current version PostgreSQL 9.2 for lack of information.
Plain SQL
The simple query could look like this:
SELECT max(buildid)
FROM app_build
WHERE buildid !~ '\d+_\d+_\d+_\d+$' -- to exclude old format
GROUP BY substring(buildid, '^[^_]+')
ORDER BY substring(buildid, '^[^_]+');
The WHERE condition used a regular expression:
buildid !~ '\d+_\d+_\d+_\d+$'
Excludes buildid that end in 4 integer numbers divided by _.
\d .. character class shorthand for digits. Only one backslash \ in modern PostgreSQL with standard_conforming_strings = ON.
+ .. 1 or more of preceding atom.
$ .. As last character: anchored to the end of the string.
There may be a cheaper / more accurate way, you did not properly specify the format.
GROUP BY and ORDER BY extract the the string before the first occurrence of _ with substring() as app name to group and order by. The regexp explained:
^ .. As first character: anchor search expression to start of string.
[^_] .. Character class: any chracter that is not _.
Does the same as split_part(buildid, '_', 1). But split_part() may be faster ..
Function
If you want to write a function where the table name is variable, you need dynamic SQL. That is a plpgsql function with EXECUTE:
CREATE OR REPLACE FUNCTION getmax(_tbl regclass)
RETURNS SETOF text AS
$func$
BEGIN
RETURN QUERY
EXECUTE format($$
SELECT max(buildid)
FROM %s
WHERE buildid !~ '\d+_\d+_\d+_\d+$'
GROUP BY substring(buildid, '^[^_]+')
ORDER BY substring(buildid, '^[^_]+')$$, _tbl);
END
$func$ LANGUAGE plpgsql;
Call:
SELECT * FROM getmax('app_build');
Or if you are, in fact, using mixed case identifiers:
SELECT * FROM getmax('"App_build"');
->SQLfiddle demo.
More info on the object identifier class regclass in this related questions:
Table name as a PostgreSQL function parameter
What you want is a groupwise_max. It can be done with MAX() but the usual way is left join:
SELECT b1.buildid
FROM builds AS b1
LEFT JOIN builds AS b2 ON
split_part(b1.buildid, '_', 1)=split_part(b2.buildid, '_', 1)
AND
split_part(b1.buildid, '_', 3)::int<split_part(b2.buildid, '_', 3)::int
WHERE b2.buildid IS NULL;
But since you're using PG it can be done with DISTINCT ON ()
SELECT DISTINCT ON (split_part(buildid, '_', 1)) buildid
FROM builds
ORDER BY split_part(buildid, '_', 1),split_part(buildid, '_', 3)::int DESC
http://sqlfiddle.com/#!12/308bf/9
I have a table with a structure like this...
the_geom data
geom1 data1+3000||data2+1000||data3+222
geom2 data1+500||data2+900||data3+22232
I want to create a function that returns the records by user request.
Example: for data2, retrieve geom1,1000 and geom2, 900
Till now I created this function (see below) which works quite good but I am facing a parameter substitution problem... (you can see I am not able to substitute 'data2' for $1 in... BUT yes I can use $1 later
regexp_matches(t::text, E'(data2[\+])([0-9]+)'::text)::text)[2]::integer
MY FUNCTION
create or replace function get_counts(taxa varchar(100))
returns setof record
as $$
SELECT t2.counter,t2.the_geom
FROM (
SELECT (regexp_matches(t.data::text, E'(data2[\+])([0-9]+)'::text)::text)[2]::integer as counter,the_geom
from (select the_geom,data from simple_inpn2 where data ~ $1::text) as t
) t2
$$
language sql;
SELECT get_counts('data2') will work **but we should be able to make this substitution**:
regexp_matches(t::text, E'($1... instead of E'(data2....
I think its more a syntaxis issue, as the function execution gives no error, just interprets $1 as a string and gives no result.
thanks in advance,
A E'$1' is a string literal (using the escape string syntax) containing a dollar sign followed by a one. An unquoted $1 is the first parameter to your function. So this:
regexp_matches(t, E'($1[\+])([0-9]+)'))[2]::integer
as you've found, won't interpolate the $1 with the function's first parameter.
The regex is just a string, a string with an internal structure but still just a string. If you know that $1 will be a normal word then you could say:
regexp_matches(t, E'(' || $1 || E'[\+])([0-9]+)'))[2]::integer
to paste your strings together into a suitable regex. However, it is better to be a little paranoid, sooner or later someone is going to call your function with a string like 'ha ha (' so you should be prepared for it. The easiest way that I can think of to add an arbitrary string to a regex is to escape all the non-word characters:
-- Don't forget to escape the escaped escapes! Hence all the backslashes.
str := regexp_replace($1, E'(\\W)', E'\\\\\\1', 'g');
and then paste str into the regex as above:
regexp_matches(t, E'(' || str || E'[\+])([0-9]+)'))[2]::integer
or better, build the regex outside the regexp_matches to cut down on the nested parentheses:
re := E'(' || str || E'[\+])([0-9]+)';
-- ...
select regexp_matches(t, re)[2]::integer ...
PostgreSQL doesn't have Perl's \Q...\E and the (?q) metasyntax applies until the end of the regex so I can't think of any better way to paste an arbitrary string into the middle of a regex as a non-regex literal value than to escape everything and let PostgreSQL sort it out.
Using this technique, we can do things like:
=> do $$
declare
m text[];
s text;
r text;
begin
s = E'''{ha)?';
r = regexp_replace(s, E'(\\W)', E'\\\\\\1', 'g');
r = '(ha' || r || ')';
raise notice '%', r;
select regexp_matches(E'ha''{ha)?', r) into m;
raise notice '%', m[1];
end$$;
and get the expected
NOTICE: ha'{ha)?
output. But if you leave out the regexp_replace escaping step, you'll just get an
invalid regular expression: parentheses () not balanced
error.
As an aside, I don't think you need all that casting so I removed it. The regexes and escaping are noisy enough, there's no need to throw a bunch of colons into the mix. Also, I don't know what your standard_conforming_strings is set to or which version of PostgreSQL you're using so I've gone with E'' strings everywhere. You'll also want to switch your procedure to PL/pgSQL (language plpgsql) to make the escaping easier.
Without using plpgsql, I'm trying to urlencode a given text within a pgsql SELECT statement.
The problem with this approach:
select regexp_replace('héllo there','([^A-Za-z0-9])','%' || encode(E'\\1','hex'),'g')
...is that the encode function is not passed the regexp parameter, unless there's another way to call functions from within the replacement expression that actually works. So I'm wondering if there's a replacement expression that, by itself, can encode matches into hex values.
There may be other combinations of functions. I thought there would be a clever regex (and that may still be the answer) out there, but I'm having trouble finding it.
select regexp_replace(encode('héllo there','hex'),'(..)',E'%\\1','g');
This doesn't leave the alphanumeric characters human-readable, though.
Here is pretty short version, and it's even "pure SQL" function, not plpgsql. Multibyte chars (including 3- and 4-bytes emoji) are supported.
create or replace function urlencode(in_str text, OUT _result text) returns text as $$
select
string_agg(
case
when ol>1 or ch !~ '[0-9a-za-z:/#._?#-]+'
then regexp_replace(upper(substring(ch::bytea::text, 3)), '(..)', E'%\\1', 'g')
else ch
end,
''
)
from (
select ch, octet_length(ch) as ol
from regexp_split_to_table($1, '') as ch
) as s;
$$ language sql immutable strict;
Here's a function I wrote that handles encoding using built in functions while preserving the readability of the URL.
Regex matches to capture pairs of (optional) safe characters and (at most one) non-safe character. Nested selects allow those pairs to be encoded and re-combined returning a fully encoded string.
I've run through a test suite with all sorts of permutations (leading/trailing/only/repeated encoded characters and thus far it seems to encode correctly.
The safe special characters are _ ~ . - and /. My inclusion of "/" on that list is probably non-standard, but fits the use case I have where the input text may be a path and I want that to remain.
CREATE OR REPLACE FUNCTION oseberg.encode_uri(input text)
RETURNS text
LANGUAGE plpgsql
IMMUTABLE STRICT
AS $function$
DECLARE
parsed text;
safePattern text;
BEGIN
safePattern = 'a-zA-Z0-9_~/\-\.';
IF input ~ ('[^' || safePattern || ']') THEN
SELECT STRING_AGG(fragment, '')
INTO parsed
FROM (
SELECT prefix || encoded AS fragment
FROM (
SELECT COALESCE(match[1], '') AS prefix,
COALESCE('%' || encode(match[2]::bytea, 'hex'), '') AS encoded
FROM (
SELECT regexp_matches(
input,
'([' || safePattern || ']*)([^' || safePattern || '])?',
'g') AS match
) matches
) parsed
) fragments;
RETURN parsed;
ELSE
RETURN input;
END IF;
END;
$function$
You can use CLR and import the namespace or use the function shown in this link , this creates a T-SQL function that does the encoding.
http://www.sqljunkies.com/WebLog/peter_debetta/archive/2007/03/09/28987.aspx