I have strings which contain numbers like that:
a20cdac0_19221bdc12022bab3fe05a43df4a7dbe
I need to get only symbols after underscore symbol:
19221bdc12022bab3fe05a43df4a7dbe
Unfortunately, the amount of those symbols is always different, so I can't use just RIGHT function.
I know that probably REGEXP might help, but I can't understand how to use that exactly. Will be very grateful for the help.
Below is for BigQuery Standard SQL (using regexp)
regexp_extract(value, r'_(.*)') regexp_approach
if to apply to sample value from your question
regexp_extract('a20cdac0_19221bdc12022bab3fe05a43df4a7dbe', r'_(.*)') regexp_approach
result is
Yet, another regexp option is to use regexp_replace as in below example
regexp_replace(value, r'^.*?_', '')
Note: using split in this case is also an option unless you have more than one _ in which case you will get part between first and second _
split(value, '_')[safe_offset(1)]
Also, as you can see you need to use safe to prevent error in cases when _ is absent
You can use the split function like this
select split('a20cdac0_19221bdc12022bab3fe05a43df4a7dbe','_')[ORDINAL(2)];
https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#split
Related
I have looked through the other questions asked on excluding regex, but I was unable to find the answer to my question.
I have the SQL statement
select --(* vendor(microsoft), product(odbc) guid'12345678-1234-1234-1234-123456789012' *)-- from TAB
With regex, I want to find every single character in that string, except
--(* vendor(microsoft), product(odbc)
and
*)--
The vendor and product names (microsoft and odbc) could be anything as well, I still want to exclude it.
I don't care what kind of characters there are, or if the SQL statement is even syntactically correct. The string could look like this, and I still want to find everything, including whitespaces, excluding what I mentioned above:
{Jane Doe?= --(* vendor(micro1macro2?), product(cdb!o) 123$% --(**) *)-- = ?
So far, I have this expression:
(--\(\* vendor\(.*\), product\(.*?\))|(\*\)--)
Which seems to work in finding what I want to exclude https://regex101.com/r/rMbYHz/204. However, I'm unable to negate it.
Does replace() do what you want?
select replace(replace(t.col, '--(* vendor(microsoft), product(odbc)', ''
), '*)--', ''
)
I'm using SQL Server 2008 and need to strip out quite a bit of data within a string. Because of the nature and variability of the string, I think I'm needing to use multiple, nested REPLACE commands. The problem is each REPLACE needs to build on the previous one. Here is a sample of what I'm looking at:
<Paragraph><Replacement Id="40B"><Le><Run Foreground="#FFFF0000">Treatment by </Run></Le><Op isFreeText="True"><Run Foreground="#FFFF0000">test</Run></Op><Tr><Run Foreground="#FFFF0000">. </Run></Tr></Replacement></Paragraph>
Essentially, I need it to return just the text outside of the <> brackets so for this example it would be:
Treatment by test.
Also, I wanted to mention that the strings inside the <> brackets can vary quite a bit for each row both by content and length, but it isn't relevant for what I'm needing other than making it more complex for replacing.
Here is what I've tried:
REPLACE(note,substring(note,patindex('<%>',note),CHARINDEX('>',note) - CHARINDEX('<',note) + 1),'')
And it returns:
<Replacement Id="40B"><Le><Run Foreground="#FFFF0000">Treatment by </Run></Le><Op isFreeText="True"><Run Foreground="#FFFF0000">test</Run></Op><Tr><Run Foreground="#FFFF0000">. </Run></Tr></Replacement></Paragraph>
Somehow I need to keep going with replacing each of the <> brackets but don't know how to proceed. Any help or guidance would be greatly appreciated!!!
Depending on how you have that string holding the HTML fragment available you could try to use something like:
SELECT convert(xml, '<Paragraph><Replacement Id="40B"><Le><Run Foreground="#FFFF0000">Treatment by </Run></Le><Op isFreeText="True"><Run Foreground="#FFFF0000">test</Run></Op><Tr><Run Foreground="#FFFF0000">. </Run></Tr></Replacement></Paragraph>').value('/', 'varchar(255)') as stripped
You convert it to XML and then use the built in xml parser function "value".
I am taking a mooc.
It has one assignment where a column needs to be converted to the lower case. sentence=lower(column) does the trick. But initially I thought that the syntax should be sentence=column.lower(). I looked at the documentation and I couldnt figure out the problem with my syntax. Would it be possible to explain how I could have figured out that I have a wrong syntax by searching online documentation and function definition?
I am specially confused as This link shows that string.lower() does the trick in case of the regular string python objects
from pyspark.sql.functions import regexp_replace, trim, col, lower
def removePunctuation(column):
"""Removes punctuation, changes to lower case, and strips leading and trailing spaces.
Note:
Only spaces, letters, and numbers should be retained. Other characters should should be
eliminated (e.g. it's becomes its). Leading and trailing spaces should be removed after
punctuation is removed.
Args:
column (Column): A Column containing a sentence.
Returns:
Column: A Column named 'sentence' with clean-up operations applied.
"""
sentence=lower(column)
return sentence
sentenceDF = sqlContext.createDataFrame([('Hi, you!',),
(' No under_score!',),
(' * Remove punctuation then spaces * ',)], ['sentence'])
sentenceDF.show(truncate=False)
(sentenceDF
.select(removePunctuation(col('sentence')))
.show(truncate=False))
You are correct. When you are working with a string, if you want to convert it to lowercase, you should use str.lower().
And if you check the String page in the Python Documentation, you will see it has a lower method that should work as you expect:
a_string = "StringToConvert"
a_string.lower() # "stringtoconvert"
However. in the Spark example you provided, in your function removePunctuation you are NOT working with a singlestring, you are working with a Column. And a Column is a different object than a string, that is way you should use a method that works with a Column.
Specifically, you are working with this pyspark sql method. The next time you are in doubt on which method you need to implement, double check the datatype of your objects. Also, if you check the list of imports, you will see it is calling the lower method from pyspark.sql.functions
This is how i managed to do it:
lowered = lower(column)
np_lowered = regexp_replace(lowered, '[^\w\s]', '')
trimmed_np_lowered = trim(np_lowered)
return trimmed_np_lowered
return trim(lower(regexp_replace(column, "\p{Punct}", ""))).alias('sentence')
I have the following regular expression:
WHERE A.srvc_call_id = '40750564' AND REGEXP_LIKE (A.SRVC_CALL_DN, '[^TEST]')
The row that contains 40750564 has "TEST CALL" in the column SRVC_CALL_DN and REGEXP_LIKE doesn't seem to be filtering it out. Whenever I run the query it returns the row when it shouldn't.
Is my regex pattern wrong? Or does SQL not accept [^whatever]?
The carat anchors the expression to the start of a string. By enclosing the letters T, E, S & T in square brackets you're searching, as barsju suggests for any of these characters, not for the string TEST.
You say that SRVC_CALL_DN contains the string 'TEST CALL', but you don't say where in the string. You also say that you're looking for where this string doesn't match. This implies that you want to use not regexp_like(...
Putting all this together I think you need:
AND NOT REGEXP_LIKE (A.SRVC_CALL_DN, '^TEST[[:space:]]CALL')
This excludes every match from your query where the string starts with 'TEST CALL'. However, if this string may be in any position in the column you need to remove the carat - ^.
This also assumes that the string is always in upper case. If it's in mixed case or lower, then you need to change it again. Something like the following:
AND NOT REGEXP_LIKE (upper(A.SRVC_CALL_DN), '^TEST[[:space:]]CALL')
By upper-casing SRV_CALL_DN you ensure that you're always going to match but ensure that your query may not use an index on this column. I wouldn't worry about this particular point as regular expressions queries can be fairly poor at using indexes anyway and it appears as though SRVC_CALL_ID is indexed.
Also if it may not include 'CALL' you will have to remove this. It is best when using regular expressions to make your match pattern as explicit as possible; so include 'CALL' if you can.
Try with '^TEST' or '^TEST.*'
Your regexp means any string not starting with any of the characters: T,E,S,T.
But your case is so simple, starts with TEST. Why not use a simple like:
LIKE 'TEST%'
I need to remove leading zeros from a string field in an Access database that is destroyed and recreated every time it is used within a C# program. Most string libraries (even SQL ones) include a Trim function to remove leading or following whitespace. Unfortunately, Access does not seem to have a LTrim(string s, char[] trimChars) or something similar. To get around this, I concocted this monstrosity:
Replace(LTrim(Replace(ADDRNO,'0', ' ')),' ', '0')
But this resulted in an undefined function reference for Replace, even though it is obviously an Access function.
What I am looking for is a way to trim these zeros, either by getting the JET engine to let me use the Replace function or by some other method entirely.
EDIT: Fixed syntax of Replace function. Problem still persists.
I suggest
Val(ADDRNO)
It will return the number portion without the leading zeros.
I think it's just the order of your parameters that is wrong:
debug.? Replace("My string", "i", "o") -> "My Strong"
You can use Trim and Replace.
I'm not sure what context you are running this but this seems to show the parameter order is different and uses double quotes instead of single quotes(I haven't used Access in awhile so maybe it doesn't matter), also try square brackets on column name:
http://www.techonthenet.com/access/functions/string/replace.php
Replace(LTrim(Replace([ADDRNO], "0", " "))," ", "0")
If that gives the same error just try the replace function by itself to narrow down the problem:
Replace ("alphabet", "a", "e")
If this works then you know the Replace function works, and there is some other issue.
Edit: If it doesn't work at all, then Replace is likely a VBA function available only in the Access application, and is not part of Jet. You could try some combination of Left/Right function and chop the string up, this can get quite ugly. I personally would just iterate over the record set and use C# code to modify the values. Hopefully you don't have such a large number of records that this would be a problem.