I am essentially attempting to replace all of the footnotes in a large text. There are various reasons I am doing this in Objective-C, so please assume that constraint.
Every footnote beings with this: [Footnote
Every footnote ends with this: ]
There can be absolutely anything between those two markers, including line breaks. However, there will never be ] between them.
So, essentially I want to match [Footnote, then match anything except ], until ] is matched.
This is the closest I have been able to get to identifying all of the footnotes:
NSString *regexString = #"[\\[][F][o][o][t][n][o][t][e][^\\]\n]*[\\]]";
Using this regular expression manages to identify 780/889 footnotes. It also appears that none of those 780 are false alarms. The only ones it appears to miss are those footnotes that have line breaks in them.
I have spent a lengthly amount of time on www.regular-expressions.info, specifically on the page about dots (http://www.regular-expressions.info/dot.html). This has helped me to create the above regular expressions, but I have not truly figured out how to include any character or line break, except right bracket.
Using the following regular expression instead manages to capture all of the footnotes, but it captures way too much text, because * is greedy: (?s)[\\[][F][o][o][t][n][o][t][e].*[\\]]
Here is some sample text that the regular expression is run on:
<p id="id00082">[Footnote 1: In the history of Florence in the early part of the XVIth century <i>Piero di Braccio Martelli</i> is frequently mentioned as <i>Commissario della Signoria</i>. He was famous for his learning and at his death left four books on Mathematics ready for the press; comp. LITTA, <i>Famiglie celebri Italiane</i>, <i>Famiglia Martelli di Firenze</i>.—In the Official Catalogue of MSS. in the Brit. Mus., New Series Vol. I., where this passage is printed, <i>Barto</i> has been wrongly given for Braccio.</p>
<p id="id00083">2. <i>addi 22 di marzo 1508</i>. The Christian era was computed in Florence at that time from the Incarnation (Lady day, March 25th). Hence this should be 1509 by our reckoning.</p>
<p id="id00084">3. <i>racolto tratto di molte carte le quali io ho qui copiate</i>. We must suppose that Leonardo means that he has copied out his own MSS. and not those of others. The first thirteen leaves of the MS. in the Brit. Mus. are a fair copy of some notes on physics.]</p>
<p id="id00085">Suggestions for the arrangement of MSS treating of particular subjects.(5-8).</p>
When you put together the science of the motions of water, remember to include under each proposition its application and use, in order that this science may not be useless.--
[Footnote 2: A comparatively small portion of Leonardo's notes on water-power was published at Bologna in 1828, under the title: "_Del moto e misura dell'Acqua, di L. da Vinci_".]
In this example there are two footnotes and some non-footnote text. The first footnote, as you can see, contains two line breaks inside it. The second one contains no line breaks.
The first regular expression I mentioned above will manage to capture Footnote 2 in this example text, but it will not capture Footnote 1 because it contains line breaks.
Any improvements on my regular expression would be most appreciated.
Try
#"\\[Footnote[^\\]]*\\]";
This should match across newlines. No need to put a single character into a character class, either.
As a commented, multiline regex (without string escapes):
\[ # match a literal [
Footnote # match literal "Footnote"
[^\]]* # match zero or more characters except ]
\] # match ]
Inside a character class ([...]), the caret ^ takes on a different meaning; it negates the contents of the class. So [ab] matches a or b, whereas [^ab] matches any character except a or b.
Of course, if you have nested footnotes, this will malfunction. A text like [Footnote foo [footnote bar] foo] will match from the beginning until bar]. To avoid this, change the regex to
#"\\[Footnote[^\\]\\[]*\\]";
so neither opening nor closing brackets are allowed. Then of course, you only match the innermost Footnotes and will have to apply the same regex twice (or more, depending on the maximum level of nesting) to the entire text, "peeling back" layer by layer.
Related
All.
I am used to programming VBA in Excel, but am new to the structures in Word.
I am working through a library of text files to update them. Many of them are either OCR documents, or were manually entered.
Each has a recurring pattern, the most common of which is unnecessary carriage returns.
For example, I am looking at several text files where there is a double return after each line. A search and replace of all double carriage returns removes all paragraph distinctions.
However, each line is approximately 30 characters long, and if I manually perform the following logic, it gives me a functional document.
If there is a double carriage return after 30+ characters, I replace them with a space.
If there were less than 30 characters prior to the double return, I replace them with a single return.
Can anyone help me with some rudimentary code that would help me get started on that? I could then modify it for each "pattern" of text documents I have.
e.g.
In this case, there are more than
thirty characters per line. And I
will keep going to illustrate this
example.
This would be a new paragraph, and
would be separated by another of
the single returns.
I want code that would return:
In this case, there are more than thirty character returns. And I will keep going to illustrate this example.
This would be a new paragraph, and would be separated by another of the single returns.
Let me know if anyone can throw something out that I can play with!
You can do this without code (which RegEx requires), simply using Word's own wildcard Find/Replace tools, where:
Find = ([!^13]{30,})[^13]{1,}
Replace = \1^32
and, to clean up the residual multi-paragraph breaks:
Find = [^13]{2,}
Replace = ^p
You could, of course, record the above as a macro...
Here is a RegEx that might work for you:
(\n\n)(?<!\.(\n\n))
The substitution is just a plain space, you can try it out (and modify / tweak it) here: https://regex101.com/r/zG9GPw/4
This 'pattern' tells the RegEx engine to look for the newline character \n which occurs x2 like this \n\n (worth noting this is from your question and might be different in your files, e.g. could be \r\n) and it assumes that a valid line break will be proceeded by a full stop: \..
In RegEx the full stop symbol is a single character wild card so it needs to be escaped with the '\' (n and r are normal characters, escaping them tells the RegEx engine they represent newline and return characters).
So... the expression is looking for a group of x2 newline characters but then uses a negative look-behind to exclude any matches where the previous character was a full stop.
Anyway, it's all explained on the site:
Here is how you could do a RegEx find and replace using NotePad++ (I'm not sure if it comes with RegEx or if a plugin is needed, either way it is easy). But you can set a location, filters (to target specific file types), and other options (such as search in sub-directories).
Other than that, as #MacroPod pointed out you could also do this with MS Word, document by document, not using any code :)
My program makes calculations on physics vectors and it allows copy/pasting from websites and then tries to parse them into the x, y, and z components automatically. I've come across one website (http://mathinsight.org/cross_product_examples) that has (3,−3,1). While that looks normal, that minus is actually not recognized by VB. Visually, it is longer than the normal minus (− and -), but return the same Unicode of 45. This picture shows the Unicode for every character (I added a minus in front of the first 3 for comparison) in the Textbox. Also, from this website, I had to use Ctrl+c because right clicking shows that this is not simple HTML.
One is valid (the first), but the second gives VB fits as shown below. Either it won't compile (shown by the blue line below) or a simple assignment (the second one) wrecks havok on my form.
I have tried using
vectorString.Replace("–", "-")
and pasting in the longer dash for the target string and a normal keystroke dash as the replacement, but nothing happens. I'm guessing that since they both have the same Unicode.
Is there some way to convert the longer, invalid dash into the one recognized by VB? I tried using dash symbol that Word likes to replace the minus sign with and it comes up as Unicode 150. So, apparently there are at least three different kinds of dashes. Any thoughts?
The character from Math Insight is U+2212, minus sign. The character you tried using in your Replace call is U+2013, en dash. That's why your replace didn't work.
Beyond the standard ASCII hyphen (-, U+0045), there are two common dashes: the en dash (–, U+2013) and the em dash (—, U+2014). There is also a figure dash (‒, U+2012), but it is not as common.
Let me apologize first. I've been fighting this SO editor for an hour. Sorry for the lousy formatting.
If I have a regex that matches a given input, then I put that regex into the positive look-behind wrapper, won't it still match the input it matched before?
For example, this input :
(NSString*)
will register a match with this regex:
\(\w*\*\)
I have confirmed this on gskinner.com. When I put that regex into the look-behind wrapper like so
(?<=\(\w*\*\))....
with this as the input:
(NSString*)help
I do not receive the word help as a return.
This leads me to think I just plainly don't understand the look-behind concept. I watched a tutorial on this concept, but I am at a loss as to why this won't work. If I want to match:
(NSString*)
and return the next word, how can I go about that?
You have a space as the last character of the look behind, but your input has no space before "help". Also, there is no colon character before the input text, yet your look behind requires one.
Remove the space and the colon:
(?<=\(\w*\*\))\w+
Note that many regex engines disallow variable length look behinds, so a work around is to limit the.number of characters in the word to some large number, eg 99:
(?<=\(\w{1,99}\*\))\w+
A bit new to regexp and looking for some help understanding some of the capabilities. I'am currently trying to select some sets of data that start with a word followed by a space and then several possible words.
Example 1:
I am basically looking to select data such as Product1 green, Product1 red, Product1 blue (green, red or blue basically) but not:
xyz Product1, Product1 black, Product1 white, Product1 garbage red.
I have tried to the following queries with not much success:
Where regexp_like(item, 'Product1 [green | red | blue]');
Where regexp_like(item, 'Product1 [green, red, blue]');
Where regexp_like(item, '^Product1 [green, red, blue]');
Hypothetically, does anybody know of a way I could also implement an 'AND', for example selecting items which contain the words green and red in the same attribute.
Example 2:
Similar situation, but trying to match a word after a punctuation
Where regexp_like (job, 'Commerce [[:punct:]] .*');
With this query I am looking to select jobs which have
Commerce - test
Commerce : abcdefg
These queries are not working as I would expect them to and I'm not able to quite figure out why. I am assuming I have misunderstood the construct of these regular expressions.
Any help / explanations would be greatly appreciated!
For the first, try the following
WHERE REGEXP_LIKE(ITEM, '^Product1.*(green|red|blue)')
or
WHERE REGEXP_LIKE(ITEM, '^Product1 (green|red|blue)')
or
WHERE REGEXP_LIKE(ITEM, '^Product1 +(green|red|blue)')
depending on what you expect after the Product1 - the first case allows zero or more characters of any kind, the second requires that there be a single space after Product1, and the third requires one or more blanks after Product1.
Not sure where you're going exactly on the second one. If you really want strings that begin with 'Commerce', followed by a space, followed by a punctuation character, another space, and then anything, try
WHERE REGEXP_LIKE(JOB, '^Commerce [:punct:] .*');
If instead of a punctuation character you're looking for either ':' or '-', try
WHERE REGEXP_LIKE(JOB, '^Commerce [:-] .*');
I'm no great expert on regular expressions but I'll try to offer some explanations:
^ requires that the following element be at the beginning of the string. Thus, in the first case ^Product1 means "'Product1' must be at the the start of the string".
In regular expressions parentheses are used to group expressions, so in the first case (green|red|blue) are grouped together.
| is a logical OR, so (green|red|blue) means "must be one of 'green' or 'red' or 'blue'".
Square brackets are used for character classes. You can use either predefined classes, such as :punct: or :space:, or you can make up your own as in [:-]. During regular expression interpretation a square bracket character class, no matter how long, represents a single character in the string being matched. So in the regular expression ^Commerce [:-] .* the character class [:-] means "look for either a colon or a dash". If you want to indicate that you expect multiple occurrences of characters in the class, one after another, use one of the repetition operators (* or +) after the class - so [abc]* would match all of abcabcabc.
Also keep in mind that in a regular expression every character means something, so you can't use whitespace to make regular expressions more legible because the whitespace becomes something that will be looked for when the expression is interpreted.
Share and enjoy.
Edit
Didn't notice your question about AND earlier. A simple way to AND together multiple expressions is to just put them one after another. To look for (green|red|blue), followed by a space, followed by (green|red|blue) a simple expression would be
WHERE REGEXP_LIKE(ITEM, '^Product1 (green|red|blue) (green|red|blue)')
If potentially multiple spaces were to be allowed between the colors
WHERE REGEXP_LIKE(ITEM, '^Product1 (green|red|blue) +(green|red|blue)')
could be used.
Resistance is useless.
According to TkDocs:
The "1.0" here represents where to insert the text, and can be read as "line 1, character 0". This refers to the first character of the first line; for historical conventions related to how programmers normally refer to lines and characters, line numbers are 1-based, and character numbers are 0-based.
I hadn't heard of this convention before, and I can't find anything relevant on Google. Can anyone explain this to me please?
I think you're referring to Tk's text widget. The man page says:
Lines are numbered from 1 for consistency with other UNIX programs that use this numbering scheme.
Although, I'm not sure which Unix tools it's talking about.
Update:
As mentioned in the comments, it looks like a lot of unix text manipulation tool starts line numbering at 1. And tcl/tk having a unix origin, it makes sense to be as compatible as possible with the underlying OS environment.
It really is nothing more than convention, but here is a suggestion.
Character positions are generally thought of in the same way as a Java iterator, which is a "pointer" to a position between two characters. Thus the first character is the one after index position 0. Substrings are taken between two inter-character positions, for instance.
Line positions on the other hand are generally thought of more in the way of a .NET enumerator, which is a "pointer" to the item itself, not to a position in between. Thus the first line is the line at position 1.