Extract only the text from PDF files with CGPDFScanner - objective-c

Question

There are a number of questions (some answered and others not) about extracting simple text from PDF files. Stackoverflow has been helpful to point out that the PDF Adobe documentation is very clear to detect objects during parsing: i.e. one should use 'BT' and 'ET' PDF reference Operators to construct the callbacks when using CGPDFScanner.
The apple documentation shows a callback example:
static void op_BT (CGPDFScannerRef s, void *info) {
const char *name;
if (!CGPDFScannerPopName(s, &name))
return;
printf("BT /%s\n", name);
}
And, among other CGPDFScanner commands, the above call-back is set-up by first creating:
myTable = CGPDFOperatorTableCreate();
CGPDFOperatorTableSetCallback (myTable, "BT", &op_BT);
All good so far, but the Apple documentation doesn't appear to help low-to-intermediate programmers like me to understand the next step: Beyond identifying the text block (presumably between BT and BE callbacks?), what few steps/lines are needed during/in/outside the callback to capture the identified text block into a NSString?
Many thanks.

Answer 1

The first thing you should do is download the PDF reference. These days that's an ISO standard, but you can download the Acrobat SDK (http://www.adobe.com/devnet/acrobat.html) which contains an Adobe copy that will serve you just as well.
Read chapter 9. It'll teach you that on the one hand you need to understand text operators (Tj, ', ", TJ) and on the other hand you need to understand fonts and encodings.
The text operators are the operators that you can intercept that add "strings" to the PDF document; while all text operators must appear between BT and ET blocks, intercepting these BT and ET blocks by itself isn't going to do much for you I think.
Fonts are important because they will define how the bytes used by those operators correspond to actual (Unicode) characters. So if you want to derive the meaning of the bytes you get from the PDF file, you need to know how to use fonts to derive that meaning.
Some additional points:
Don't assume BT and ET correspond to an actual text block or paragraph as you may know it from an application such as InDesign or Word. One text block may contain a whole page or a single character (or nothing).
There are also text state operators that determine how the text is going to be shown on the page. There are ways for example to draw invisible text; you may or may not wish to extract that type of text. If you don't, you'll need to support enough text state operators that you can tell the difference.
Not a small task :)
Update after looking at sample PDF
Because in comments the question was refined to indicate text extraction of a specific type of PDF file, let me add a little additional information.
1) Looking at the PDF file you reference, you won't be able to skip the font/encoding problem. The fonts in the sample PDF file are subsetted which means that you don't have "cleartext" in the PDF page description but instead indexes that have to be mapped through the encoding of the fonts used to get meaningful text.
2) Extracting the text is possible, if you look at the following output from pdfToolbox (warning, I'm affiliated rather heavily with this tool):
<page id="33">
<words>
<word txt="Senator">
<parts>
<part tlh="28.3481" tlv="868.534" trh="55.4455" trv="868.534" blh="28.3481" blv="859.902" brh="55.4455" brv="859.902"></part>
</parts>
</word>
<word txt="House,">
<parts>
<part tlh="57.5305" tlv="868.534" trh="82.123" trv="868.534" blh="57.5305" blv="859.902" brh="82.123" brv="859.902"></part>
</parts>
</word>
<word txt="85">
<parts>
<part tlh="84.208" tlv="868.534" trh="92.548" trv="868.534" blh="84.208" blv="859.902" brh="92.548" brv="859.902"></part>
</parts>
</word>
There are undoubtedly other tools which can give a similar (or better) result, so extracting the text by itself should be doable.
The big problem is going to be finding the text you're interested in in the right order. The extraction I used here gives the text of each "word" and it's position (bounding box) on the page. When I look through the XML when you get to the table, the challenge is going to be which text belongs to which table cell, where rows and columns end etc...
In a way this problem is harder than the problem of simply detecting lines of text because you're dealing with a pretty dense table (and where my problem was largely one-dimensional (gathering everything on the same line) this problem is two-dimensional.