I need to write a REGEXP_REPLACE query for a spark.sql() job.
If the value, follows the below pattern then only, the words before the first hyphen are extracted and assigned to the target column 'name', but if the pattern doesn't match, the entire 'name' should be reported.
Pattern:
Values should be hyphen delimited. Any values can be present before the first hyphen (be it numbers,
alphabets, special characters or even space)
First hyphen should be exactly followed by 2 words, separated by hyphen (it can only be numbers,
alphabets or alphanumeric) (Note: Special characters & blanks are not allowed)
Two words should be followed by one or more digits, followed by hyphen.
Last portion should be only one or more digits.
For Example:
if name = abc45-dsg5-gfdvh6-9890-7685, output of REGEXP_REPLACE = abc45
if name = abc, output of REGEXP_REPLACE = abc
if name = abc-gf5-dfg5-asd5-98-00, output of REGEXP_REPLACE = abc-gf5-dfg5-asd5-98-00
I have
spark.sql("SELECT REGEXP_REPLACE(name , '-[^-]+-\\w{2}-\\d+-\\d+$','',1,1,'i') AS name").show();
But it does not work.
Use
^([^-]*)(-[a-zA-Z0-9]+){2}-[0-9]+-[0-9]+$
See proof. Replace with $1. If $1 does not work, use \1. If \1 does not work use \\1.
EXPLANATION
--------------------------------------------------------------------------------
^ the beginning of the string
--------------------------------------------------------------------------------
( group and capture to \1:
--------------------------------------------------------------------------------
[^-]* any character except: '-' (0 or more
times (matching the most amount
possible))
--------------------------------------------------------------------------------
) end of \1
--------------------------------------------------------------------------------
( group and capture to \2 (2 times):
--------------------------------------------------------------------------------
- '-'
--------------------------------------------------------------------------------
[a-zA-Z0-9]+ any character of: 'a' to 'z', 'A' to
'Z', '0' to '9' (1 or more times
(matching the most amount possible))
--------------------------------------------------------------------------------
){2} end of \2 (NOTE: because you are using a
quantifier on this capture, only the LAST
repetition of the captured pattern will be
stored in \2)
--------------------------------------------------------------------------------
- '-'
--------------------------------------------------------------------------------
[0-9]+ any character of: '0' to '9' (1 or more
times (matching the most amount possible))
--------------------------------------------------------------------------------
- '-'
--------------------------------------------------------------------------------
[0-9]+ any character of: '0' to '9' (1 or more
times (matching the most amount possible))
--------------------------------------------------------------------------------
$ before an optional \n, and the end of the
string
Related
In an Oracle SQL query there is a field with the following content (example):
{"ID Card": 0.29333333333333333} or {"Speedtest": 0.8166666666666667}
Can I use RegEx, for example, to output the field in the query so that only the numbers and the period remain?
Example:
Select ID, CREATEDFORMAT, INSERT_TS, regexp_substr(SCORE, '[^0-9]') xSCORE FROM MYTABLE
But with the [^ 0-9] I only have the numbers without a point.
If you are using Oracle Database 12.1.0.2 or higher and the number you are trying to parse out is always in a JSON object, you can use the JSON_VALUE function to pull the information out.
Query
WITH
sample_data
AS
(SELECT '{"ID Card": 0.29333333333333333}' AS sample_val FROM DUAL
UNION ALL
SELECT '{"Speedtest": 0.8166666666666667}' FROM DUAL)
SELECT s.sample_val, json_value (s.sample_val, '$.*') AS number_val
FROM sample_data s;
Result
SAMPLE_VAL NUMBER_VAL
____________________________________ ______________________
{"ID Card": 0.29333333333333333} 0.29333333333333333
{"Speedtest": 0.8166666666666667} 0.8166666666666667
Use
REGEXP_SUBSTR(SCORE, '[-+]?[0-9]*\.?[0-9]+')
See proof
Explanation
--------------------------------------------------------------------------------
[-+]? any character of: '-', '+' (optional
(matching the most amount possible))
--------------------------------------------------------------------------------
[0-9]* any character of: '0' to '9' (0 or more
times (matching the most amount possible))
--------------------------------------------------------------------------------
\.? '.' (optional (matching the most amount
possible))
--------------------------------------------------------------------------------
[0-9]+ any character of: '0' to '9' (1 or more
times (matching the most amount possible))
use: REGEXP_SUBSTR (s.sample_val, '[+-]?[0-9]+[\.]?[0-9]+')
see this demo: https://dbfiddle.uk/?rdbms=oracle_18&fiddle=76c2b3be1d7d266f217d6b0541478c17
result:
SAMPLE_VAL NUMBER_VAL
---------------------------------- --------------------
{"ID Card": 0.29333333333333333} 0.29333333333333333
{"Speedtest": 0.8166666666666667} 0.8166666666666667
{"texts": 12.3456} 12.3456
{"texts": -65} -65
This is a change to #Ryszard Czech's post.
How can i make a regex that string should contain char and number. if its just letter or just number it should return me false
Eg:
123swift -> true
swift123 -> true
1231 -> false
swift -> false
My regex:
[a-z]|[0-9]
Use
^(?=.*?[A-Za-z])(?=.*?[0-9])[0-9A-Za-z]+$
Or, a presumably more efficient version:
^(?=[^A-Za-z]*[A-Za-z])(?=[^0-9]*[0-9])[0-9A-Za-z]+$
See proof.
Expanation:
NODE EXPLANATION
--------------------------------------------------------------------------------
^ the beginning of the string
--------------------------------------------------------------------------------
(?= look ahead to see if there is:
--------------------------------------------------------------------------------
.*? any character except \n (0 or more times
(matching the least amount possible))
--------------------------------------------------------------------------------
[A-Za-z] any character of: 'A' to 'Z', 'a' to 'z'
--------------------------------------------------------------------------------
) end of look-ahead
--------------------------------------------------------------------------------
(?= look ahead to see if there is:
--------------------------------------------------------------------------------
.*? any character except \n (0 or more times
(matching the least amount possible))
--------------------------------------------------------------------------------
[0-9] any character of: '0' to '9'
--------------------------------------------------------------------------------
) end of look-ahead
--------------------------------------------------------------------------------
[0-9A-Za-z]+ any character of: '0' to '9', 'A' to 'Z',
'a' to 'z' (1 or more times (matching the
most amount possible))
--------------------------------------------------------------------------------
$ before an optional \n, and the end of the
string
Unfortunately \b regular expression character doesn't work in Oracle.
As a workaround I found following expression:
(^|\s|\W)(100100|100101|100102|100103)($|\s|\W)
(see: The missing \b regular expression special character in Oracle.), but in the test string data:
Test string 100100/100101, ABC-DEF, 100102 100103 test data abc100100 100100abc.
values 100101 and 100103 are not matched, while I am expecting them to be matched like it is the case of \b expression.
Is there any way to make it working? I am using Oracle 11g.
I would be appreciated for any help.
EDIT:
My goal is to tag all matches. The output that I am expecting is:
Test string [ddd]100100[/ddd]/[ddd]100101[/ddd], ABC-DEF, [ddd]100102[/ddd] [ddd]100103[/ddd] test data abc100100 100100abc.
In this purpose I am using following statement:
regexp_replace(p_text,'(^|\s|\W)(' || l_ids || ')($|\s|\W)', '\1[ddd]\2[/ddd]\3');
Where:
l_ids - list of ids separated by |, id can contain number, letters, underscores and dashes
p_text - input text
EDIT 2:
In the above test string value 100100 should not be matched in the word abc100100 as well as 100100abc.
Assuming -
chr(1) does not appear in the text
Any character that is not in [a-zA-Z0-9] is considered as a delimiter (e.g. /)
with t (p_text) as (select 'Test string 100100/100101, ABC-DEF, 100102 100103 test data abc100100 100100abc.' from dual)
select replace
(
regexp_replace
(
regexp_replace
(
p_text
,'([a-zA-Z0-9]+)'
,chr(1) || '\1' || chr(1)
)
,chr(1) || '(100100|100101|100102|100103)' || chr(1)
,'[ddd]\1[/ddd]'
)
,chr(1)
)
from t
Test string [ddd]100100[/ddd]/[ddd]100101[/ddd], ABC-DEF,
[ddd]100102[/ddd] [ddd]100103[/ddd] test data abc100100 100100abc.
Why does this fail for the first character in oracle sql?
select DECODE( TRANSLATE('1','123',' '), NULL, 'number','contains char') from dual
This works because 1 is the second digit
select DECODE( TRANSLATE('1','4123',' '), NULL, 'number','contains char') from dual
But this fails because 4 is the first digit
select DECODE( TRANSLATE('4','423',' '), NULL, 'number','contains char') from dual
First let's take a look at translate function definition:
TRANSLATE(expr, from_string, to_string): TRANSLATE returns expr with all
occurrences of each character in from_string replaced by its corresponding
character in to_string. Characters in expr that are not in from_string are not replaced.
If expr is a character string, then you must enclose it in single quotation marks.
The argument from_string can contain more characters than to_string. In this case,
the extra characters at the end of from_string have no corresponding characters
in to_string. If these extra characters appear in char, then they are removed
from the return value.
i.e. TRANSLATE(some_string,'123','abc'): 1 will be replaced by a, 2 by b, 3 by c(I will use arrow -> instead of "replaced by" further)
Now let's take a look at our examples:
TRANSLATE('1','123',' '): 1 -> " ", 2->nothing, 3->nothing.
(nothing means removed from the return value, see definition)
Result of above function is string consisted of whitespace - " "
TRANSLATE('1','4123',' '): 4 -> " ", 1->nothing, 2->nothing, 3->nothing
Result of the above function is empty string "". Oracle Database interprets the empty string as null, and if this function has a null argument, then it returns null.
TRANSLATE('4','423',' '): 4->" ", 2->nothing, 3->nothing
Result of the above function is whitespace string as in the first example.
That is why you are getting "contains char" in the first and third queries, and number in the second one
I have used REPLACE function in order to delete email addresses from hundreds of records. However, as it is known, the semicolon is the separator, usually between each email address and anther. The problem is, there are a lot of semicolons left randomly.
For example: the field:
123#hotmail.com;456#yahoo.com;789#gmail.com;xyz#msn.com
Let's say that after I deleted two email addresses, the field content became like:
;456#yahoo.com;789#gmail.com;
I need to clean these fields from these extra undesired semicolons to be like
456#yahoo.com;789#gmail.com
For double semicolons I have used REPLACE as well by replacing each ;; with ;
Is there anyway to delete any semicolon that is not preceded or following by any character?
If you only need to replace semicolons at the start or end of the string, using a regular expression with the anchor '^' (beginning of string) / '$' (end of string) should achieve what you want:
with v_data as (
select '123#hotmail.com;456#yahoo.com;789#gmail.com;xyz#msn.com' value
from dual union all
select ';456#yahoo.com;789#gmail.com;' value from dual
)
select
value,
regexp_replace(regexp_replace(value, '^;', ''), ';$', '') as normalized_value
from v_data
If you also need to replace stray semicolons from the middle of the string, you'll probably need regexes with lookahead/lookbehind.
You remove leading and trailing characters with TRIM:
select trim(both ';' from ';456#yahoo.com;;;789#gmail.com;') from dual;
To replace multiple characters with only one occurrence use REGEXP_REPLACE:
select regexp_replace(';456#yahoo.com;;;789#gmail.com;', ';+', ';') from dual;
Both methods combined:
select regexp_replace( trim(both ';' from ';456#yahoo.com;;;789#gmail.com;'), ';+', ';' ) from dual;
regular expression replace can help
select regexp_replace('123#hotmail.com;456#yahoo.com;;456#yahoo.com;;789#gmail.com',
'456#yahoo.com(;)+') as result from dual;
Output:
| RESULT |
|-------------------------------|
| 123#hotmail.com;789#gmail.com |