this is my code.
%spark.pyspark
jdbc_write(spark, spark.sql("""
SELECT
Global_Order_Number__c
, Infozeile__c
FROM STAG.SF_CASE_TRANS
"""), JDBC_URLS['xyz_tera_utf8'], "DEV_STAG.SF_CASE", "abc", "1234")
I want to exclude every character in the Infozeile__c field which are not a-z, A-Z, 0-9.
Is their any function which is able to do this?
Apply regexp_replace() to the column in your query:
regexp_replace(Infozeile__c, '[^a-zA-Z0-9]', '') as Infozeile__c
The regex [^a-zA-Z0-9] is a negated character class, meaning any character not in the ranges given. The replacement is a blank, effectively deleting the matched character.
If you're expecting lots of characters to be replaced like this, it would be a bit more efficient to add a +, which means "one or more", so whole blocks of undesirable characters are removed at a time.
regexp_replace(Infozeile__c, '[^a-zA-Z0-9]+', '')
Related
I need to find rows where the phone number field contains unexpected characters.
Most of the values in this field look like:
123456-7890
This is expected. However, we are also seeing character values in this field such as * and #.
I want to find all rows where these unexpected character values exist.
Expected:
Numbers are expected
Hyphen with numbers is expected (hyphen alone is not)
NULL is expected
Empty is expected
Tried this:
WHERE phone_num is not like ' %[0-9,-,' ' ]%
Still getting rows where phone has numbers.
from https://regexr.com/3c53v address you can edit regex to match your needs.
I am going to use example regex for this purpose
select * from Table1
Where NOT REGEXP_LIKE(PhoneNumberColumn, '^[+]*[(]{0,1}[0-9]{1,4}[)]{0,1}[-\s\./0-9]*$')
You can use translate()
...
WHERE translate(Phone_Number,'a1234567890-', 'a') is NOT NULL
This will strip out all valid characters leaving behind the invalid ones. If all the characters are valid, the result would be NULL. This does not validate the format, for that you'd need to use REGEXP_LIKE or something similar.
You can use regexp_like().
...
WHERE regexp_like(phone_num, '[^ 0123456789-]|^-|-$')
[^ 0123456789-] matches any character that is not a space nor a digit nor a hyphen. ^- matches a hyphen at the beginning and -$ on the end of the string. The pipes are "ors" i.e. a|b matches if pattern a matches of if pattern b matches.
Oracle has REGEXP_LIKE for regex compares:
WHERE REGEXP_LIKE(phone_num,'[^0-9''\-]')
If you're unfamiliar with regular expressions, there are plenty of good sites to help you build them. I like this one
I have a requirement where the persons can have more than one first name and i need to convert the name into first characters from the 2 or more names with capital letters.
Examples:
Srinivas Kalyan ,Sai Kishore
if the above is one having 4 first names then i need to display as below
S K S K
The comma is also be replaced and take all the first characters
2. Srinivas-Kalyan Sai Kishore
For the above name the value should be as
S S K
since Srinivas-Kalyan is considered as one name
Also the name can be in small letters
srinivas kalyan sai kishore
For this
S K S K
This has to be in oracle
Tried the below regex_replace which is working fine in sql developer but in the application it is changing into space
replace(trim(regexp_replace(to_char(regexp_replace(initcap(regexp_replace(regexp_replace
(FIRST_NAME, '[0-9]', ''), '( *[[:punct:]])', '')), '([[:lower:]]| )')), '(.)', '\1 ')),',',null)
The missing letter in your output is caused by this regex for character removal:
( *[[:punct:]])
This will turn Kalyan ,Sai into KalyanSai, which will be treated as one word by the rest of the process, and so you will not have the S of Sai in your output.
I would suggest this shorter expression:
trim(upper(regexp_replace(
regexp_replace(first_name, '([[:alpha:]])(-|[[:alpha:]])+', '\1'),
'.*?([[:alpha:]]|$)', ' \1'
)))
Explanation
([[:alpha:]])(-|[[:alpha:]])+ -> \1
This replaces any sequence of letters (or hyphen) with the first of those. So it reduces words to their initial letter.
.*?([[:alpha:]]|$) -> (space)\1
This replaces anything that precedes the next letter, with a space. As no letter will be skipped (because .*? is non-greedy) this effectively replaces all non-letter sequences with a space. To also replace the non-letters at the very end (which do not precede a letter), the special case $ (end-of-string) is added as alternative.
After these two steps there just remains to:
Upper case everything
Remove blanks at the start and end of the result with trim
I find the advantage of this method is that it does not use any other class than [[:alpha]]. Digits, punctuation, lowercase -- or whatever else -- does not need to be identified explicitly, as it is just the negation of [[:alpha]]. The only exception that has to be made, is for the hyphen.
As some names might include some other non-letters, like a quote, you might want to add such characters as well in the first regular expression.
how to remove special characters from Teradata columns.
I am having columns values other than characters A-Z and numbers 1-9, how to remove theses column values as we don't know the exact special character. Can any one share the command for it?
You can use a regular expression, '[^0-9a-z]' to find characters which don't match a list of characters:
regexp_replace(col, '[^0-9a-z]', '', 1, 0, 'i')
I have a list of values coming from a PGSQL database that looks something like this:
198
199
1S
2
20
997
998
999
C1
C10
A
I'm looking to parse this field a bit into individual components, which I assume would take two regexp_replace function uses in my SQL. Essentially, any non-numeric character that appears before numeric ones needs to be returned for one column, and the other column would show all non-numeric characters appearing AFTER numeric ones.
The above list would then be split into this layout as the result from PG:
I have created a function that strips out the non-numeric characters (the last column) and casts it as an Integer, but I can't figure out the regex to return the string values prior to the number, or those found after the number.
All I could come up with so far, with my next to non-existant regex knowledge, was this: regexp_replace(fieldname, '[^A-Z]+', '', 'g'), which just strips out anything not A-Z, but I can;t get to to work with strings before numeric values, or after them.
For extracting the characters before the digits:
regexp_replace(fieldname, '\d.*$', '')
For extracting the characters after the digits:
regexp_replace(fieldname, '^([^\d]*\d*)', '')
Note that:
if there are no digits, the first will return the original value and then second an empty string. This way you are sure that the concatenation is equal to the original value in this case also.
the concatenation of the three parts will not return the original if there are non-numerical characters surrounded by digits: those will be lost.
This also works for any non-alphanumeric characters like #, [, ! ...etc.
Final SQL
select
fieldname as original,
regexp_replace(fieldname, '\d.*$', '') as before_s,
regexp_replace(fieldname, '^([^\d]*\d*)', '') as after_s,
cast(nullif(regexp_replace(fieldname, '[^\d]', '', 'g'), '') as integer) as number
from mytable;
See fiddle.
This answer relies on information you delivered, which is
Essentially, any non-numeric character that appears before numeric
ones needs to be returned for one column, and the other column would
show all non-numeric characters appearing AFTER numeric ones.
Everything non-numeric before a numeric value into 1 column
Everything non-numeric after a numeric value into 2 column
So there's assumption that you have a value that has a numeric value in it.
select
val,
regexp_matches(val,'([a-zA-Z]*)\d+') AS before_numeric,
regexp_matches(val,'\d+([a-zA-Z]*)') AS after_numeric
from
val;
Attached SQLFiddle for a preview.
I try to extract the '930' from 'EM 930' with following Regexp
REGEXP_REPLACE(info,'^[:space:]*[ABCDEFGHIJKLMNOPQRSTUVWXYZ]*[:space:]*([0-9]+)[:space:]*$','\1')
But it returns me the original string.
An idea why ?
Subsidiary question:
Why does the "\1" returned the original string when the pattern is not matched ? I expected it to return NULL, as in my other regexp experiences (eg Perl).
Who I can re-write this in a performant way so that I get of wel the matched string of well NULL ?
Your space character class was not exactly correct. If we change [:space:] to [[:space:]], your regexp_replace works as you expect:
REGEXP_REPLACE(info, '^[[:space:]]*[ABCDEFGHIJKLMNOPQRSTUVWXYZ]*[[:space:]]*([0-9]+)[[:space:]]*$','\1')
For the sake of succinctness, we could use the upper character class, [[:upper:]], for [ABCDEFGHIJKLMNOPQRSTUVWXYZ]. This changes the function invocation to:
regexp_replace(info, '^[[:space:]]*[[:upper:]]*[[:space:]]*([0-9]+)[[:space:]]*$','\1')
Or escape characters could be used in lieu of character classes:
\s space
\w word character
\d digit character
regexp_replace(info, '^\s*\w*\s*(\d+)\s*$','\1')
Explanation:
Since your malformed character class, [:space:], does not match the space that exists between 'EM' and '930', your search by parameter does not match any characters in the source parameter.
Your search by parameter, '^[[:space:]]*[ABCDEFGHIJKLMNOPQRSTUVWXYZ]*[[:space:]]*([0-9]+)[[:space:]]*$', is anchored to the beginning and end of the column, info, thus it can only match the column, info, one time at most.
In your case, there is no match and the character group, '\1' which is associated with '([0-9]*)', has no value.
Consequently, no characters are replaced and you are left with original value of the column, info, 'EM 930'.
interesting variations to better understand this function:
-If your corrected function invocation had no pattern_to_replace_by parameter, '\1', then a NULL would be returned:
regexp_replace(info, '^\s*\w*\s*(\d+)\s*$' ) FROM dual;
-Since you have a pattern_to_replace_by parameter, '\1', and now it has the matching character group, the repeating digit group is returned:
930