How wildcards in sqlite work. Or how LIKE operator matches.
For examle lets say:
1: LIKE('s%s%', 's12s12')
2: LIKE('asdaska', '%sk%')
In 1st example what % matches after 1st s, and how it decides to continue matching % or s after %.
In 2nd example if s matches first then FALSE returned.
Both examples return TRUE. From my Programming knowledge I came up with that LIKE function is some like a recursive function that when 2 possibilities appear function calls itself with 2 different params and uses OR between them, then obviously if one call returns true, upper function directly returns true. If it is so, then LIKE operator is quiet slow to use on large DBs.
P.S. There is one more '_' wildcard which matches exactly one character
I couldnt find any detailed documentation of LIKE operator.
% matches zero or more characters, _ matches exactly one.
Your first pattern 's%s%' would match, 'ss', 's1s', 's1111s', 'ss1111', etc. etc.
However if you wrote 's_s_' it would match 's1s1', but none of the above.
Related
I am currently facing a Regex problem which apparently I cannot find an answer to.
My Regex is embedded in a teradata SQL of the form:
REGEXP_SUBSTR(column, 'regex_pattern')
I want to find the first appearance of any number except if it appears at the end of the string.
For Example:
"YEL2X30" -> "2"
"YEL19XYZ05" -> "19"
"YELLOW05" -> ""
I tried it with '[0-9]+(?!$)/' but this returns me a blank String always.
Thanks in Advance!
Shot in the dark here since I'm unfamiliar with teradata and the supported SQL-functionality. However, reading the docs on the REGEXP_SUBSTR() function it seems like you may want to use the 3rd and 4th possible argument along with a slightly different regular expression:
[0-9]+(?![0-9]|$)
Meaning: 1+ Digits that are not followed by either the end of the string or another digit.
I'd believe the following syntax may work now to retrieve the 1st appearance of any number from the matching results:
REGEXP_SUBSTR(column, '[0-9]+(?![0-9]|$)', 1, 1)
The 3rd parameter states from which position in the source-string we need to start searching whereas the 4th will return the 1st match from any possible multiple matches (is how I read the docs). For example: abc123def456ghi789 whould return 123.
Fiddling around in online IDE's gave me that:
CREATE TABLE TBL (TST varchar(100));
INSERT INTO TBL values ('YEL2X30'), ('YEL19XYZ05'), ('YELLOW05'), ('abc123def456ghi789');
SELECT REGEXP_SUBSTR(TST, '[0-9]+(?![0-9]|$)', 1, 1) as 'RESULTS' FROM TBL;
Resulted in:
RESULTS
2
19
NULL
123
NOTE: I also noticed that leaving out the 3rd and 4th parameter made no difference since they will default back to 1 without explicitly mentioning them. I tested this over here.
Possibly the simplest way is to look for digits followed by a non-digit. Then keep all the digits:
regexp_substr(regexp_substr(column, '[0-9]+[^0-9]'), '[0-9]+')
Am trying to determine how one attempts to identify, in Snowflake SQL, if a product code begins with three letters.
Suggestions?
I did just try: LEFT(P0.PRODUCTCODE,3) NOT LIKE '[a-zA-Z]%' but it didn't work.
Thanks folks
You can use REGEXP_LIKE to return a boolean value indicating whether or not your string matched the pattern you're interested in.
In your case, something like REGEXP_LIKE(string_field_here, '[a-zA-Z]{3}.*')
Breaking down the regular expression pattern:
[a-zA-Z]: Only match letter characters, both upper and lowercase
{3}: Require three of those letters
.*: Allow any number of any characters after those three letters
Note: in many cases, you would need to specifically indicate the beginning/ending of the string in the pattern, but Snowflake's implementation handles that for you. From the docs:
The function implicitly anchors a pattern at both ends (i.e. ''
automatically becomes '^$', and 'ABC' automatically becomes '^ABC$').
To match any string starting with ABC, the pattern would be 'ABC.*'.
You can try running these examples:
SELECT REGEXP_LIKE('abc', '[a-zA-Z]{3}.*') AS _abc,
REGEXP_LIKE('123', '[a-zA-Z]{3}.*') AS _123,
REGEXP_LIKE('abc123', '[a-zA-Z]{3}.*') AS _abc123,
REGEXP_LIKE('123abc', '[a-zA-Z]{3}.*') AS _123abc
I don't fully understand, why the results are different here. Does :ov apply only to <left>, so having found the longest match it wouldn't do anything else?
my regex left {
a | ab
}
my regex right {
bc | c
}
"abc" ~~ m:ex/<left><right>
{put $<left>, '|', $<right>}/; # 'ab|c' and 'a|bc'
say '---';
"abc" ~~ m:ov/<left><right>
{put $<left>, '|', $<right>}/; # only 'ab|c'
Types of adverbs
It's important to understand that there are two different types of regex adverbs:
Those that fine-tune how your regex code is compiled (e.g. :sigspace/:s, :ignorecase/:i, ...). These can also be written inside the regex, and only apply to the rest of their lexical scope within the regex.
Those that control how regex matches are found and returned (e.g. :exhaustive/:ex, :overlap/:ov, :global/:g). These apply to a given regex matching operation as a whole, and have to be written outside the regex, as an adverb of the m// operator or .match method.
Match adverbs
Here is what the relevant adverbs of the second type do:
m:ex/.../ finds every possible match at every possible starting position.
m:ov/.../ finds the first possible match at every possible starting position.
m:g/.../ finds the first possible match at every possible starting position that comes after the end of the previous match (i.e., non-overlapping).
m/.../ finds the first possible match at the first possible starting position.
(In each case, the regex engine moves on as soon as it has found what it was meant to find at any given position, that's why you don't see additional output even by putting print statements inside the regexes.)
Your example
In your case, there are only two possible matches: ab|c and a|bc.
Both start at the same position in the input string, namely at position 0.
So only m:ex/.../ will find both of them – all the other variants will only find one of them and then move on.
:ex will find all possible combinations of overlapping matches.
:ov acts like :ex except that it limits the search algorithm by constraining it to find only a single match for a given starting position, causing it to produce a single match for a given length. :ex is allowed to start from the very beginning of the string to find a new unique match, and so it may find several matches of length 3; :ov will only ever find exactly one match of length 3.
Documentation:
https://docs.perl6.org/language/regexes
Exhaustive:
To find all possible matches of a regex – including overlapping ones – and several ones that start at the same position, use the :exhaustive (short :ex) adverb
Overlapping:
To get several matches, including overlapping matches, but only one (the longest) from each starting position, specify the :overlap (short :ov) adverb:
I have database have thousand of unknow string they may be emails ,phonenum
BUT they are not for me mean they are not email or cell num for me they are only string for me but i want their common pattern so here is the string for example purposes
link to example click here
now what i want is this file out put if pattern matcehs 3 time here what i am doing is
DECLARE #strs2 nvarchar(255)
DECLARE #patternTable table(
id int ,
order by p.pat
but my example return this
485-2889
485-2889
) 485-2889
) 485-2889
.aol.com/aol/search?
.aol.com/aol/search?
gmail.com
gmail.com
but i want to add this for pattern
[a-zA-Z 0-9] [a-zA-Z 0-9] [a-zA-Z 0-9] - 485-2889
for gmail
[a-zA-Z 0-9] [a-zA-Z 0-9]# gmail.com
First of all, this is much more work than it might seem.
As far as I can say it's going to be method with heavy processing (and probably not something you want to do with a cursor in SQL (cursors are sort of bad in terms of efficiency).
You have to define a way for your code to identify a pattern. You will also have to work in priorities where a set of strings matches multiple patterns. For instance if you implement following pattern criteria (in your example):
BK-M18B-48
BK-M18B-52
BK-M82B-44
BK-M82S-38
BK-M82S-44
BK-R50B-58
BK-R50B-62
.....
should generate BK-[A-Z]-[0-9][0-9][A-Z]-[0-9][0-9]
Then next set can have multiple patterns as a result:
fedexcarepackage#outlook.com (example added for explanations)
fedexcarepackage#office.com
fedexcourierexpress#pisem.net
fedexcouriers#gmail.com ( another example added for explanations)
.....
Can generate :
fedexc%#%.% (as you said)
fedexc%#% (depending on processing)
fedexc[A-Z][A-Z]....%#%[A-Z]....[A-Z].[A-Z][A-Z][A-Z] (alphanumeris with '%' to compensate for length difference)
in addition to that if you take away fedexcarepackage#outlook.com from string list you get 1 additional pattern that you probably don't want to have:
fedexc%#%i%.% (because they have 'i' somewhere between the '#' and '.' (dot)
Anyway, that is something you will have to consider with your design.
I'll give you some basic logic you can work with:
Create a functions to identify each distinct pattern (1 pattern / function). For instnace, 1 function to check for static pieces of string (and attaching wildcards); Another to detect [A-Z],[0-9] patterns that match your conditions for this pattern to be valid; more if needed for different patterns.
Create a function to test a string with your pattern. So say you have 4 string, you find a pattern when comparing first 2 of them. Then you use this function to test if pattern applies to 3rd and 4th strings.
Create a function to test if 2 patterns are mutually exclusive. For instance 'PersonA#yahoo.%' and 'PersonA#%.net' patterns are not mutually exclusive, if they were both tested to be true. 'Person%#yahoo.com' and 'PersonB#yahoo.com' are mutually exclusive (both patterns cannot be true, so 1 is redundant.
Create a function to combine patterns that are NOT mutually exclusive (probably includes the use of function in 2nd and 3rd point). So 'PersonA#yahoo.%' and 'PersonA#%.net' can be combined into 'PersonA#%.%'
Once you have that setup, loop through each text line, and compare Current line to the next against each pattern criteria. Record any patterns you find (in a variable dedicated to that criteria, (don't mix them just yet).
Next comes the hardest part, safest way is to compare each pattern you find against each of the strings, to rule out the ones that don't apply to all strings. However, you could probably work out a way to combine patterns (in the same category) without cross checking
Finally, after you narrowed own your pattern list to 1 pattern per pattern type. Combine them into 1 or eliminate the ones
Keep in mind that in your pattern detection functions, you'll probably have to test each line multiple times and combine patterns. Some pseudo code to demonstrate:
Function CompareForStringMatches (String s1, String s2){ -- it should return a possible pattern found.
Array/List pattern;
int patternsFound=0;
For(i = 0, to length of shorter string){
For(x = 0, to length of shorter string){
if(longerString.contains(shorterString.substring(from i, to x)){
--record the pattern somewhere as:
pattern[patternsFound] = Replace(longerString, shorterString.Substring(from i, to x), '%') --pattern = longerString with substring replaced with '%' sign
patternsFound = patternsFound+1;
}
}
}
--After loops make another loop to check (partial) patterns against each other to eliminate patterns that are part of a larger pattern
--for instance Comparing 'random#asd.com' and 'sundom#asd.com' the patterns below should be found:
---compare'%andom#asd.com' and '%ndom#asd.com' and eliminate the first pattern, because both are valid, but second pattern includes the first one.
--You will have a lot of similar matches, but if you do this, you should end up with only a few patterns.
--after first cycle of checks do another one to combine patterns, where possible(for instance if you compare 'random#asd.com' and 'sundom#asd.net' you will end up with these 2 patterns'%ndom#asd.com' and 'Random#asd.%'.
--Since these patterns are true (because they were found during a comparison) you can combine them into '%ndom#asd.%'
--when you combine/eliminate all patterns, you should only have 1 left
return pattern[only pattern left];
}
PS: You can do things, much more efficiently, but if you have no idea where to start out, you probably need to do it the long way and work on improvements from first working prototypes.
Edit/Update
I suggest you make a wildcard detection method and then apply other patter checks you implement before it.
Wildcard detection for comparison of 2 strings (pseudo code), heavy processing version :
Compare 2 strings, check if every possible segment of shorter string is within longer:
for(int i = 0; i<shorterString.Length;i++){
for(int x = 0; i<shorterString.Length;i++){
if(longerString.contains(shorterString.substring(i,x))){ --from i to x
possiblePattern.Add(longerString.replace(shorterString.substring(i,x),'*')
--add to pattern list
}
}
--Next compare partal matches and eliminate ones that are a part of larger pattern
--So '*a#gmail.com' and '*na#yahoo.com' comparison should eliminate '*na#gmail.com', because if shorter pattern (with more symbols removed) is valid, then similar one with an extra symbol is part of it
--When that is done, combine remaining matches if there's more than 1 left.
--Remember, all patterns are valid if your first loop was correct, so '*#gmail.com' and 'personA#*.com' can be combined into '*#*.com
}
As for the alphanumeric detection. I would suggest you start by checking length of all strings. If they are the same, run the wildcard pattern detection method (for all of them). When done ONLY look for patern matches in wildcards.
So, You'll get a pattern like BK-*-* from wildcard detection run. On second iteration loop take 2 strings and only extract sub-strings that are represented by wildcard characters (use an array or an equivalent to store sub-strings, make sure not to combine both wildcards of a single string into 1 string).
So if you compare with pattern found above (BK-*-*) :
BK-M18B-48
BK-M18B-52
You should get following string sets to process after eliminating static characters:
Set 1:M18B and 48
Set 2:M18B and 52
Compare each character to opposite string in same position and check if characters match your category (like if String1[0].isaLetter AND String2[0].isaLetter). If they do add that 1 character to a pattern, if not either:
Add a wildcard character (will lead to pattern like BK-[A-Z]*[0-9][0-9]-[0-9][0-9]. If you do this combine adjacent wildcard characters to 1.
Pattern is false and you should abbort the ch'eck returning no patterns.
Use this basic logic to loop through strings, create (and store!!!!) patterns for each set of 2 strings. Loop through patterns, with wildcard detection (possibly a lighter version) to combine/eliminate paterns. So if you get patterns like '#yahoo.com' and '#gmail.com' from different sets of strings you should combine them into '#.com'
Keep in mind there's lots of room for optimization here.
I need to match two ipaddress/hostname with a regular expression:
Like 20.20.20.20
should match with 20.20.20.20
should match with [http://20.20.20.20/abcd]
should not match with 20.20.20.200
should not match with [http://20.20.20.200/abcd]
should not match with [http://120.20.20.20/abcd]
should match with AB_20.20.20.20
should match with 20.20.20.20_AB
At present i am using something like this regular expression: "(.*[^(\w)]|^)20.20.20.20([^(\w)].*|$)"
But it is not working for the last two cases. As the "\w" is equal to [a-zA-Z0-9_]. Here I also want to eliminate the "_" underscore. I tried different combination but not able to succeed. Please help me with this regular expression.
(.*[_]|[^(\w)]|^)10.10.10.10([_]|[^(\w)].*|$)
I spent some more time on this.This regular expression seems to work.
I don't know which language you're using, but with Perl-like regular expressions you could use the following, shorter expression:
(?:\b|\D)20\.20\.20\.20(?:\b|\D)
This effectively says:
Match word boundary (\b, here: the start of the word) or a non-digit (\D).
Match IP address.
Match word boundary (\b, here: the end of the word) or a non-digit (\D).
Note 1: ?: causes the grouping (\b|\D) not to create a backreference, i.e. to store what it has found. You probably don't need the word boundaries/non-digits to be stored. If you actually need them stored, just remove the two ?:s.
Note 2: This might be nit-picking, but you need to escape the dots in the IP address part of the regular expression, otherwise you'd also match any other character at those positions. Using 20.20.20.20 instead of 20\.20\.20\.20, you might for example match a line carrying a timestamp when you're searching through a log file...
2012-07-18 20:20:20,20 INFO Application startup successful, IP=20.20.20.200
...even though you're looking for IP addresses and that particular one (20.20.20.200) explicitly shouldn't match, according to your question. Admittedly though, this example is quite an edge case.