Consider the following function arguments (they are already extracted of the function):
Monkey,"Blue Monkey", "Red, blue and \"Green'", 'Red, blue and "Green\''
Is there a way to extract arguments to get the following array ouput using regexp and stripping white spaces:
[Monkey, "Blue Monkey", "Red, blue and \"Green'", 'Red, blue and "Green\'']
I'm stuck using this RegExp which is not permisive enough:
/(("[^"]+"|[^\s,]+))/g
This looks a little nasty but it works:
/(?:"(?:[^\x5C"]+|\x5C(?:\x5C\x5C)*[\x5C"])*"|'(?:[^\x5C']+|\x5C(?:\x5C\x5C)*[\x5C'])*'|[^"',]+)+/g
I used \x5C instead of the plain backslash character \ as too much of those can be confusing.
This regular expression consists of the parts:
"(?:[^\x5C"]+|\x5C(?:\x5C\x5C)*[\x5C"])*" matches double quoted string declarations
'(?:[^\x5C']+|\x5C(?:\x5C\x5C)*[\x5C'])*' matches single quoted string declarations
[^"',]+ matches anything else (except commas).
The parts of "(?:[^\x5C"]+|\x5C(?:\x5C\x5C)*[\x5C"])*" are:
[^\x5C"]+ matches anything except the backspace and quote character
\x5C(?:\x5C\x5C)*[\x5C"] matches proper escape sequences like \", \\, \\\", \\\\, etc.
Not sure exactly what you're seeking, nor yet how to do this in SQL, but isn't something like this sufficient:
(Using python as an example)
import re
x = '''Monkey, "Blue Monkey", "Red, blue and "Green\\"", 'Red, blue and "Green\\'\''''
l = re.split(',\s*',x)
print x
for a in l:
print a
Related
i have a column in openrefine, which has cells with content like:
This dog is a great dog.
This cat is a great cat,
i would like to remove the words dog, cat from the end of each cell (if punctuation could be removed also, it would be great).
i have tried with
\bdog\s*$
but i get errors, or no replacement done
I am using openrefine 3.3.
value.replace(\bdog|\bcat\s*$,'')
error i get:
Parsing error at offset 14: Missing number, string, identifier, regex, or parenthesized expression
desired output:
This dog is a great
This cat is a great
also, it would be great if i could also remove all characters in the end like " : , . (actually i am looking for a regex to cluster publishers -librarian data) so if you could suggest words i should remove from the end of the cells i would be grateful
I combined Ettore answer with the split() function value.split(' ')[-1] that select the last part word of a string.
The results is :
replace(value,value.split(' ')[-1],'') + value.split(' ')[-1].replace(/cat|dog/,'')
where
replace(value,value.split(' ')[-1],'') select your string expect the last work
value.split(' ')[-1].replace(/cat|dog/,'') replace the last word with nothing if it contains cat or dog.
Note that the expression is working because of the punctuation at the end of the string. Not a perfect solution but you may be able to build something from here.
Hi may i know what does the below query means?
REGEXP_REPLACE(number,'[^'' ''-/0-9:-#A-Z''[''-`a-z{-~]', 'xy') ext_number
part 1
In terms of explaining what the function function call is doing:
It is a function call to analyse an input string 'number' with a regex (2nd argument) and replace any parts of the string which match a specific string. As for the name after the parenthesis I am not sure, but the documentation for the function is here
part 2
Sorry to be writing a question within an answer here but I cannot respond in comments yet (not enough rep)
Does this regex work? Unless sql uses different syntax this would appear to be a non-functional regex. There are some red flags, e.g:
The entire regex is wrapped in square parenthesis, indicating a set of characters but seems to predominantly hold an expression
There is a range indicator between a single quote and a character (invalid range: if a dash was required in the match it should be escaped with a '\' (backslash))
One set of square brackets is never closed
After some minor tweaks this regex is valid syntax:
^'' ''\-\/0-9:-#A-Z''[''-a-z{-~]`, but does not match anything I can think of, it is important to know what string is being examined/what the context is for the program in order to identify what the regex might be attempting to do
It seems like it is meant to replaces all ASCII control characters in the column or variable number with xy.
[] encloses a class of characters. Any character in that class matches. [^] negates that, hence all characters match, that are not in the class.
- is a range operator, e.g. a-z means all characters from a to z, like abc...xyz.
It seams like characters enclosed in ' should be escaped (The second ' is to escape the ' in the string itself.) At least this would make some sense. (But for none of the DBMS I found having a regexp_replace() function (Postgres, Oracle, DB2, MariaDB, MySQL), I found something in the docs, that would indicate this escape mechanism. They all use \, but maybe I missed something? Unfortunately you didn't tag which DBMS you're actually using!)
Now if you take an ASCII table you'll see, that the ranges in the expression make up all printable characters (counting space as printable) in groups from space to /, 0 to 9, : to #, etc.. Actually it might have been shorter to express it as '' ''-~, space to ~.
Given the negation, all these don't match. The ones left are from NUL to US and DEL. These match and get replaced by xy one by one.
I have a column eventDate which contains trailing spaces. I am trying to remove them with the PostgreSQL function TRIM(). More specifically, I am running:
SELECT TRIM(both ' ' from eventDate)
FROM EventDates;
However, the trailing spaces don't go away. Furthermore, when I try and trim another character from the date (such as a number), it doesn't trim either. If I'm reading the manual correctly this should work. Any thoughts?
There are many different invisible characters. Many of them have the property WSpace=Y ("whitespace") in Unicode. But some special characters are not considered "whitespace" and still have no visible representation. The excellent Wikipedia articles about space (punctuation) and whitespace characters should give you an idea.
<rant>Unicode sucks in this regard: introducing lots of exotic characters that mainly serve to confuse people.</rant>
The standard SQL trim() function by default only trims the basic Latin space character (Unicode: U+0020 / ASCII 32). Same with the rtrim() and ltrim() variants. Your call also only targets that particular character.
Use regular expressions with regexp_replace() instead.
Trailing
To remove all trailing white space (but not white space inside the string):
SELECT regexp_replace(eventdate, '\s+$', '') FROM eventdates;
The regular expression explained:
\s ... regular expression class shorthand for [[:space:]]
- which is the set of white-space characters - see limitations below
+ ... 1 or more consecutive matches
$ ... end of string
Demo:
SELECT regexp_replace('inner white ', '\s+$', '') || '|'
Returns:
inner white|
Yes, that's a single backslash (\). Details in this related answer:
SQL select where column begins with \
Leading
To remove all leading white space (but not white space inside the string):
regexp_replace(eventdate, '^\s+', '')
^ .. start of string
Both
To remove both, you can chain above function calls:
regexp_replace(regexp_replace(eventdate, '^\s+', ''), '\s+$', '')
Or you can combine both in a single call with two branches.
Add 'g' as 4th parameter to replace all matches, not just the first:
regexp_replace(eventdate, '^\s+|\s+$', '', 'g')
But that should typically be faster with substring():
substring(eventdate, '\S(?:.*\S)*')
\S ... everything but white space
(?:re) ... non-capturing set of parentheses
.* ... any string of 0-n characters
Or one of these:
substring(eventdate, '^\s*(.*\S)')
substring(eventdate, '(\S.*\S)') -- only works for 2+ printing characters
(re) ... Capturing set of parentheses
Effectively takes the first non-whitespace character and everything up to the last non-whitespace character if available.
Whitespace?
There are a few more related characters which are not classified as "whitespace" in Unicode - so not contained in the character class [[:space:]].
These print as invisible glyphs in pgAdmin for me: "mongolian vowel", "zero width space", "zero width non-joiner", "zero width joiner":
SELECT E'\u180e', E'\u200B', E'\u200C', E'\u200D';
'' | '' | '' | ''
Two more, printing as visible glyphs in pgAdmin, but invisible in my browser: "word joiner", "zero width non-breaking space":
SELECT E'\u2060', E'\uFEFF';
'' | ''
Ultimately, whether characters are rendered invisible or not also depends on the font used for display.
To remove all of these as well, replace '\s' with '[\s\u180e\u200B\u200C\u200D\u2060\uFEFF]' or '[\s]' (note trailing invisible characters!).
Example, instead of:
regexp_replace(eventdate, '\s+$', '')
use:
regexp_replace(eventdate, '[\s\u180e\u200B\u200C\u200D\u2060\uFEFF]+$', '')
or:
regexp_replace(eventdate, '[\s]+$', '') -- note invisible characters
Limitations
There is also the Posix character class [[:graph:]] supposed to represent "visible characters". Example:
substring(eventdate, '([[:graph:]].*[[:graph:]])')
It works reliably for ASCII characters in every setup (where it boils down to [\x21-\x7E]), but beyond that you currently (incl. pg 10) depend on information provided by the underlying OS (to define ctype) and possibly locale settings.
Strictly speaking, that's the case for every reference to a character class, but there seems to be more disagreement with the less commonly used ones like graph. But you may have to add more characters to the character class [[:space:]] (shorthand \s) to catch all whitespace characters. Like: \u2007, \u202f and \u00a0 seem to also be missing for #XiCoN JFS.
The manual:
Within a bracket expression, the name of a character class enclosed in
[: and :] stands for the list of all characters belonging to that
class. Standard character class names are: alnum, alpha, blank, cntrl,
digit, graph, lower, print, punct, space, upper, xdigit.
These stand for the character classes defined in ctype.
A locale can provide others.
Bold emphasis mine.
Also note this limitation that was fixed with Postgres 10:
Fix regular expressions' character class handling for large character
codes, particularly Unicode characters above U+7FF (Tom Lane)
Previously, such characters were never recognized as belonging to
locale-dependent character classes such as [[:alpha:]].
It should work the way you're handling it, but it's hard to say without knowing the specific string.
If you're only trimming leading spaces, you might want to use the more concise form:
SELECT RTRIM(eventDate)
FROM EventDates;
This is a little test to show you that it works.
Tell us if it works out!
If your whitespace is more than just the space meta value than you will need to use regexp_replace:
SELECT '(' || REGEXP_REPLACE(eventDate, E'[[:space:]]', '', 'g') || ')'
FROM EventDates;
In the above example I am bounding the return value in ( and ) just so you can easily see that the regex replace is working in a psql prompt. So you'll want to remove those in your code.
SELECT replace((' devo system ') ,' ','');
It gives: devosystem
A tested one that works like a charm:
UPDATE company SET name = TRIM (BOTH FROM name) where id > 0
Unicode have categories of characters. Some are alpha numeric. Some are punctuation.
What about if I want to know whether a word belongs to keyword or not
For example,
A,a,b,c, tend to belong to words. So is Ƈ,Ǝ,ǟ, so are all chinese characters.
Sentences like
Hello World, I "like" (to) eat ƇƎǟ and 款开源 ©
Have keywords:
Hello
World
I
like
to
eat
ƇƎǟ
款
开
源
Here, , (),© are not word characters and hence should just be ignored and use.
© doesn't count as punctuation either. '©'.IsPunctuation returns false in vb.net but I want to get rid of that too.
Now I want to make a program that can split sentences into keywords. For that I need to know which characters are word characters and which one is not.
Is there a vb.net function for that?
Do it the other way round: use IsLetter for your test. Or better yet, use regular expressions to split your string by words:
Dim str = "Hello World, I ""like"" (to) eat ƇƎǟ and 款开源 ©"
Dim wordPattern As New Regex("\p{L}+")
For Each match in wordPattern.Matches(str))
Console.WriteLine(match)
Next
Here, \p{L} matches any word character. However, the above matches “款开源” in a single rather than in separate matches since there is no separator between the characters.
u need to deal with "keycodes"
like if u only want letters [a-z]
then
for(c>='a' && c<='z'){
}
or
for(c>=97 && C<=122){
}
Why does the space make all the difference?
select * from beds where id~'.*Extra large.* (Red).*';
and
select * from beds where id~'.*Extra large.*(Red).*';
The first one returned nothing and the second acted as I wanted. An example of what I want matched is:
"Extra large" (Red) {2012 model}
I thought the first would work since there is a space after (Red)?
EDIT:Even if I escape the brackets with '\' I still can't have a space there.
The problem is that you have not escaped your brackets around "Red". Your regex should be:
'.*Extra large.* \(Red\).*'
This makes the brackets literal brackets, but without escaping them they create a regex group (and not characters to be matched).
Your first regex grouped the characters Red and required a space to precede that group Red, so it would match "... Red...", but there is a bracket in your input before Red, so it doesn't match.
Your second regex accepts any character(s) (via .*) before Red, so it matches.
This is because you're not escaping the ().
The brackets around "Red" create a group and are not included in the match. This
is the reason why the regexp without the whitespace works.
The .* in the regexp without the whitespace matches " (, then comes Red and after that ) {2012 model}. The brackets are matched by the .* operators.
The .* in the regexp with the whitespace matches " and the ( is not included in the pattern.
So the right pattern would be this:
.*Extra large.*\(Red\).*