I created a code on GoogleAppScript with ORC to get text from a PDF file on Google Drive, using the getFileById() but the problem is that this file is an Adobe PDF Forms type and the code reads only the texts that are not in the fields that were edited in the form. Does anyone have any suggestions on how I can get this?
The file that I used as an example: http://foersom.com/net/HowTo/data/OoPdfFormExample.pdf (Please, fill the file and save on your drive to test it)
The PDF on my Drive:
PDF file image
This is the result:
values shown when code is executed
Here is my code:
function extractTextFromPDF() {
var fileId = '[File ID here]';
const ss = SpreadsheetApp.getActiveSpreadsheet()
//Get all PDF files:
const fileID = DriveApp.getFileById(fileId);
var blob = DriveApp.getFileById(fileId).getBlob()
var resource = {
title: blob.getName(),
mimeType: blob.getContentType(),
};
// Enable the Advanced Drive API Service
var file = Drive.Files.insert( resource, blob, { ocr: true, ocrLanguage: 'en' } );
//,supportsAllDrives: true
// Extract Text from PDF file
var doc = DocumentApp.openById(file.id);
var text = doc.getBody().getText();
Logger.log(text)
return text;
}
The FDF data does not need to be stored in the order you see on the page, often may be the order the fields were added. The easiest way to see the text /V(alues) in reply to /T(ext field) entries is via the FDF a user can save and send without the Source PDF. This has the added advantage that a CHECK box will be sent as /V /Yes or /V /OFF (= NOT checked) which is notoriously difficult with a binary PDF, however you need to know which text on the page was Language 1 since the author did not tag it as Deutsch!
What intrigues me most (without looking closer) is the count of attached objects only went up by 12 but there are more than that number of potentially added answers,
from a field :-) of 17.
/Lang(en-GB)
/AcroForm<</Fields[
5 0 R 7 0 R 8 0 R 9 0 R 10 0 R
11 0 R 12 0 R 13 0 R 14 0 R 16 0 R
17 0 R 18 0 R 19 0 R 20 0 R 21 0 R
22 0 R 23 0 R
]/DR 37 0 R/NeedAppearances true>>
Anyway if you dont want to do it the easy way you will need to program something to read those object chains. Personally it takes each user a few seconds to export the forms or have them sent in the background by email however most modern users dont use historic mailto: so you have to get them to web mail the form for you to press the extract button.
So here is object 5 and we can see it is for your Given Name
5 0 obj
<</Type/Annot/Subtype/Widget/F 4
/Rect[165.7 453.7 315.7 467.9]
/FT/Tx
/P 1 0 R
/T(Given Name Text Box)
/TU<FEFF004600690072007300740020006E0061006D0065>
/V <FEFF>
/DV <FEFF>
/MaxLen 40
/DR<</Font 6 0 R>>
/DA(0 0 0 rg /F3 11 Tf)
/AP<<
/N 38 0 R
>>
>>
endobj
so the entry for 004600690072007300740020006E0061006D0065 converts to First name and we can see the answer when the user sends back the form as a PDF
5 0 obj
<</AP<</N 38 0 R >>/DA(0 0 0 rg /F3 11 Tf)/DR<</Font 6 0 R >>/DV<FEFF>/F 4/FT/Tx/MaxLen 40/P 1 0 R /Rect[ 165.7 453.7 315.7 467.9]/Subtype/Widget/T(Given Name Text Box)
/TU<FEFF004600690072007300740020006E0061006D0065>/Type/Annot/V(Brunno)>>
endobj
Related
I have wrote that in notepad
a b c d e
1 1 1 1 0 0
2 1 1 0 1 0
3 1 0 1 0 1
4 0 1 1 1 0
then I save it as s.ar
I open WEKA and try to insert it but it doesn't show me that.
.ar is not a file extension that Weka recognizes.
If you meant to create an ARFF file, then please see the documentation for that file format.
Have you thought about just using CSV (comma-separated values, .csv extension) instead? You could rewrite your data like this:
ID,a,b,c,d,e
1,1,1,1,0,0
2,1,1,0,1,0
3,1,0,1,0,1
4,0,1,1,1,0
Then you can load it in Weka's Explorer (after selecting CSV data files as type from the file chooser dialog).
''AN ERROR EXISTS ON THIS PAGE. ACROBAT MAY NOT DISPLAY PAGE CORRECTLY. PLEASE CONTACT THE PERSON WHO HAS CREATED THE PDF DOCUMENT TO CORRECT THE PROBLEM.''
this error appears when trying to open pdf in Adobe reader but in crome browser it works fine.
Sampe_PDF
The reason why Adobe Acrobat Reader complains is because there indeed is trash in the content streams of the second and third page of your document, there are multiple sections like this
0 Tc
1 0 0 sc
? ? m
? ? ? ? ? ? c
? ? ? ? ? ? c
? ? ? ? ? ? c
? ? ? ? ? ? c
f
0 0 0 sc
BT
1 1 1 sc
/F1 10 Tf
? ? Td
(12) Tj
ET
All these question marks would have to be numbers to make this a valid PDF content stream. There even is one similar section with numbers filled in:
1 0 0 sc
18.5 443.384 m
18.5 447.25 21.63401 450.384 25.5 450.384 c
29.36599 450.384 32.5 447.25 32.5 443.384 c
32.5 439.51801 29.36599 436.384 25.5 436.384 c
21.63401 436.384 18.5 439.51801 18.5 443.384 c
f
0 0 0 sc
BT
1 1 1 sc
/F2 10 Tf
20 440.384 Td
(12) Tj
ET
I would assume that the producer here either used the '?' as placeholders in a template and simply forgot to fill-in these placeholders (probably what you have actually is a template, not a PDF for distribution or viewing) or added these sections with NaNs (not-a-number values) which then were serialized as question marks.
I'm currently into generating PDF documents without the use of an external library and it has been going well so far. I've written the document exposed below with a text editor (vim) and it renders the expected results using at least two PDF distinct viewers (evince & gsview, running Linux).
This document produces three squares at top of the page, coming in different sizes, widths and colors.
My question is : is there a way to merge two stream objects into a new single one or, in other words, is there a way to compose sophisticated objects starting from simple ones, so we can easily refer to these composite objects, multiple times if needed ?
In the given example, object 5 0 obj is drawing a square, and following ones are just applying colors and coordinates transformations (through a matrix).
The PDF reference manual states that multiple stream contents passed as an array to page object's /Contents parameter are concatenated and processed as a single continuous stream, which totally does the trick… as long as the document remains small and simple!
In this same example, the /Contents array is indirectly passed through object 4 0 obj, which refers three times to 5 0 R, to draw the squares.
The ideal here would be to define three differents objects, each refering to 5 0 R by themselves, then invoke only these objects, a single time each, from the Contents array.
I tried adding subarrays inside it, which could in turn be embedded into dedicated objects and referenced indirectly, but it unfortunately doesn't work. :-(
A lot of thanks to any people that could/try-to help !
PS: I'm doing it because I'm interested in the format itself and would like to produce some autogenerated documents from small scripts. Also, I'll probably embed them into a weakly powered appliance and I cannot afford relying on dozens of megabytes in dependencies.
But before this, I still tried to do that too, using PHP with TCPDF. If there's already some facilities dedicated to this that I would have missed, this is relevant to my interests too. :-)
Small.pdf (hand made PDF file)
%PDF-1.7
1 0 obj
<<
/Type /Catalog
/Pages 2 0 R
>>
endobj
2 0 obj
<<
/Type /Pages
/Count 1
/Kids [ 3 0 R ]
>>
endobj
3 0 obj
<<
/Type /Page
/MediaBox [ 0.000000 0.000000 1000.000000 1414.213562 ]
/Contents 4 0 R
>>
endobj
4 0 obj
% A simple array, just to avoid embedding it directly in /Page object (3 0 R here)
[
6 0 R 5 0 R % Red square
7 0 R 5 0 R % Green square
8 0 R 5 0 R % Blue square (tilted)
]
endobj
5 0 obj
% Draws a square, centered by default on lower left corner
<<
/Length 43
>>
stream
+20 +20 m
+20 -20 l
-20 -20 l
-20 +20 l s Q
endstream
endobj
6 0 obj
<<
/Length 63
>>
stream
/DeviceRGB CS
q
1.0 0.0 0.0 SC
2.0 w
1 0 0 -1 60 1354.213562 cm
endstream
endobj
7 0 obj
<<
/Length 49
>>
stream
q
0.0 1.0 0.0 SC
1.0 w
2 0 0 -2 190 1334.213562 cm
endstream
endobj
8 0 obj
<<
/Length 83
>>
stream
q
0.0 0.0 1.0 SC
5.0 w
0.707106781 0.707106781 -0.707106781 0.707106781 110 1250 cm
endstream
endobj
xref
0 9
0000000000 65535 f
0000000010 00000 n
0000000079 00000 n
0000000168 00000 n
0000000296 00000 n
0000000513 00000 n
0000000674 00000 n
0000000796 00000 n
0000000905 00000 n
trailer
<<
/Size 9
/Root 1 0 R
/ID [ <0000000000> <0000000001> ]
>>
startxref
01047
%%EOF
What you are looking for are form XObjects.
The pdf specification ISO 32000-1 characterizes them like this:
A form XObject is a PDF content stream that is a self-contained description of any sequence of graphics objects. A form XObject may be painted multiple times - either on several pages or at several locations on the same page - and produces the same results each time, subject only to the graphics state at the time it is invoked.
For details please read section 8.10 of the specification.
I am on ubuntu.
I have a pdf file with pages divided into a grid. Each block of the grid contains name/age/dob/photo of a candidate. some records have a watermark "disqualified"
I need to scrape his pdf, with disqualified candidates in a separate list.
Using pyPdf I was able to get individual records, but it also includes watermarked candidates.
How to detect the watermark? If I can get the coordinates of the watermark, how do I match it with the candidate?
I am open to solutions other than python pyPdf
(Actually this is not an answer but merely an analysis to bit for a comment.)
I don't know pyPdf (or any python PDF classes) myself, but here is how the watermark is created for a sample entry; based upon this, anyone knowing pyPDF well enough, may more easily advice.
The Roundup
Depending on how pyPDF (or other python PDF classes) allows access to the page content, there are two major basic approaches:
If the class returns information on content (text and image) in their order in the page content stream: The watermark image xobject is referred to right before the data of the entry. Thus, any entry preceded by the drawing of a xobject image is marked.
If otherwise the information are not given in the order indicated by the page content stream, coordinate comparison must be used which per se is quite straight forward. In that case it might be of interest that the images are inserted with a [0.1 0 0 0.1 0 0] transformation matrix in action while the text is drawn with an identity transformation matrix.
The Details
This is entry # 200; the other watermarked entry is constructed similarly:
Watermarking is done by means of an image xobject. There is but one image xobject defined for the page used by both watermarked entries:
4 0 obj
<</Type/Page/MediaBox [0 0 595 841]
/Rotate 0/Parent 3 0 R
/Resources<</ProcSet[/PDF /ImageC /ImageI /Text]
/ColorSpace 18 0 R
/ExtGState 19 0 R
/XObject 20 0 R
/Font 21 0 R
>>
/Contents 5 0 R
>>
endobj
20 0 obj
<</R17
17 0 R>>
endobj
17 0 obj
<</Subtype/Image
/ColorSpace 16 0 R
/Width 128
/Height 88
/BitsPerComponent 8
/Filter/FlateDecode/Length 463>>stream
[...]
endstream
endobj
In the content stream this xobject /R17 is inserted right before the data of the entry itself is drawn:
q 0.1 0 0 0.1 0 0 cm
[...]
q 1045 0 0 495 462.5 6510.5 cm
/R17 Do
Q
q
10 0 0 10 0 0 cm BT
0.000487366 Tc
/R10 8 Tf
1 0 0 1 86 650.75 Tm
(Sex : Male)Tj
0.000304794 Tc
-64 0 Td
(Age : 43)Tj
-0.000140686 Tc
-1 11.05 Td
(House No :)Tj
-0.00002085 Tc
1 31.95 Td
(Name :)Tj
0.00008575 Tc
/R12 7.15 Tf
25.5 17.8 Td
( 200 )Tj
ET
Q
1547.5 6475 485 535.5 re
S
q
10 0 0 10 0 0 cm BT
-0.000403137 Tc
/R14 8 Tf
1 0 0 1 145.1 708.5 Tm
(XVX0001081)Tj
0.000421651 Tc
/R14 7.05 Tf
-90.35 -14.95 Td
(Ramesh Kumar)Tj
0.000373332 Tc
/R10 7.05 Tf
-33 -12.75 Td
(Father's )Tj
0.000193787 Tc
7.3 TL
(Name)'
0.00037774 Tc
/R14 7.05 Tf
40.25 1.8 Td
(Ram Singh)Tj
0 Tc
2.5 -11.85 Td
(37)Tj
0.00137196 Tc
/R12 7.15 Tf
-5.25 13.35 Td
(:)Tj
I have a source pdf which I am modifying by adding text objects. I am using "Incremental Updates" which is mentioned in the PDF specification. But while adding text objects using this method I am making some mistakes due to which the pdf doesn't render properly in Adobe Reader 11. When the pdf is opened and I double-click on it, the added text objects get deleted. I figured out that this is due to text annotation.
Now I want to know how a new text object can be added using incremental update? How do the Contents and RC of a free text annotation have to be to maintained?
Also is it possible to disable or delete the annotation so that my problem can be avoided easily? Because I want a simple pdf, I don't want annotation options.
The source pdf I am using is here.
The modified pdf after adding text object is here.
I am not sure that source pdf is itself proper according to pdf specification.
First off let me show you how easy things are if you can use a decent PDF library. I use iTextSharp as an example but the same can also be done with others like PDFBox or PDFNet (already mentioned by #Ika in his answer):
PdfReader reader = new PdfReader(sourcePdf);
using (PdfStamper stamper = new PdfStamper(reader, targetPdfStream)) {
Font FONT = new Font(Font.FontFamily.HELVETICA, 12, Font.BOLD, new GrayColor(0.75f));
PdfContentByte canvas = stamper.GetOverContent(1);
ColumnText.ShowTextAligned(
canvas,
Element.ALIGN_LEFT,
new Phrase("Hello people!", FONT),
36, 540, 0
);
}
(Derived from the Webified iTextSharp Example StampText.cs explained in chapter 6 of iText in Action — 2nd Edition.)
(Which PDF library you choose, depends on your general requirements and available license models.)
If, in spite of the ease of use of such PDF libraries, you insist on doing it manually, here some remarks:
First you have to find the Page dictionary of the page you want to add content to. Depending on the type of PDF this already might require decompression of object streams etc. but in your sample modified1.pdf that is not necessary:
7 0 obj
<</Rotate 90
/Type /Page
/TrimBox [ 9.54 6.12 585.68 835.88 ]
/Resources 8 0 R
/CropBox [ 0 0 595.22 842 ]
/ArtBox [ 9.54 18.36 585.68 842 ]
/Contents [ 9 0 R 10 0 R 11 0 R 12 0 R 13 0 R 14 0 R 15 0 R 16 0 R ]
/Parent 6 0 R
/MediaBox [ 0 0 595.22 842 ]
/Annots 17 0 R
/BleedBox [ 9.54 6.12 585.68 835.88 ]
>>
endobj
You see the array of references to content streams. This is where you have to add new page content to. You can manipulate an existing stream or create a new stream and add it to that array.
(Most PDFs have their content stream compressed. For the general case, therefore, you'd have to decompress a stream before you can work on it. Thus, in my eyes, the easier way would be to start a new stream.)
You chose to manipulate the last referenced stream 16 0 which in your PDF is uncompressed:
16 0 obj
<</Length 37 0 R>>
stream
S 1 0 0 1 13.183 0 cm 0 0 m
[...]
0 10 -10 -0 506.238 342.629 Tm
.13333 .11765 .12157 scn
-.0002 Tc
.0006 Tw
(the Bank and branch on which cheque is drawn\).)Tj
/F1 2 Tf
-15.1279 10.9462 Td
(abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789~!##$%^&*aaaaaaaaaaaaa)Tj
/F2 1 Tf
015.1279 01.9462 Td
(ANAabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789)Tj
ET
endstream
endobj
Your additions, I gather, are the two 3-liners at the bottom which first select a font, then position the insertion point and finally print a selection of letters.
Now you say you added text abc..z and ABC...Z just for testing. But letters b j k q v etc not appearing in the pdf. The problem becomes even more visible for your second addition of letters; here only the capital 'A' and 'N' are displayed.
This is due to the fact that the fonts in question are embedded into the PDF --- fonts are embedded into PDFs to allow PDF viewers on systems which don't have the font in question, to display the PDF --- but they are not completely embedded, only the subset of characters required from that font.
Let's look for the font F2 for which only 'N' and 'A' appear:
According to the page object, the page resources can be found in object 8 0:
8 0 obj
<</Font <</F1 45 0 R /TT2 46 0 R /F2 47 0 R>>
/ExtGState <</GS2 48 0 R>>
/ProcSet [ /PDF /Text ]
/ColorSpace <</Cs6 49 0 R>>
>>
endobj
So F2 is defined in 47 0:
47 0 obj
<</Subtype /Type1
/Type /Font
/Widths [ 722 250 250 250 250 250 250 250 250 250 250 250 250 722 ]
/Encoding 52 0 R
/FirstChar 65
/FontDescriptor 53 0 R
/ToUnicode 54 0 R
/BaseFont /ILBPOB+TimesNewRomanPSMT-Bold
/LastChar 78
>>
endobj
In the referenced ToUnicode map 54 0 you see
54 0 obj
<</Length 55 0 R>>stream
/CIDInit /ProcSet findresource begin 12 dict begin begincmap /CIDSystemInfo <<
/Registry (AAAAAA+F2+0) /Ordering (T1UV) /Supplement 0 >> def
/CMapName /AAAAAA+F2+0 def
/CMapType 2 def
1 begincodespacerange <41> <4e> endcodespacerange
2 beginbfchar
<41> <0041>
<4e> <004E>
endbfchar
endcmap CMapName currentdict /CMap defineresource pop end end
endstream
endobj
In this mapping you see that only character codes 0x41 'A' and 0x4e 'N' are mapped
In your document the font is used only to print "NA" in the amount table cells and for nothing else. Thus, only those two letters 'N' and 'A' are embedded, which results in your addition with that font only outputting these letters.
Thus, to successfully add text to the page you either have to check the font ressources associated with the page for the glyphs they provide (and restrict your additions to those glyphs) or you have to add your own font resource.
As the presence of characters in the encoding often is not as easy to see as it is here (ToUnicode is optional), I would propose, you add your own font ressources. The PDF specification ISO 32000-1 explains how to do that.
Furthermore you state the x and y axis position for the text is not properly displaying in pdf. While you don't say what exactly you mean, you should be aware that in the content stream you can apply affine transformations to the coordinate system of the page, i.e. stretch, skew, rotate, and move the axis.
If you want to use the original coordinate system and not depend on the coordinates to be proper at your additions, you should add an initial content stream to the page containing a q operator (to save the current graphics state on the graphics state stack) and start your additions in a new final content stream with a Q operator (to restore the graphics state by removing the most recently saved state from the stack and making it the current state).
EDIT As a sample I applied the Java equivalent of the C# code at the top to your modified1.pdf with append mode activated. The following objects were changed or added as a result:
The page object 7 0 has been updated:
7 0 obj
<</CropBox[0 0 595.22 842]
/Parent 6 0 R
/Contents[69 0 R 9 0 R 10 0 R 11 0 R 12 0 R 13 0 R 14 0 R 15 0 R 16 0 R 70 0 R]
/Type/Page
/Resources<<
/ExtGState<</GS2 48 0 R>>
/ProcSet [/PDF /Text /ImageB /ImageC /ImageI]
/ColorSpace<</Cs6 49 0 R>>
/Font<</F1 45 0 R/F2 47 0 R/TT2 46 0 R/Xi0 68 0 R>>
>>
/MediaBox[0 0 595.22 842]
/TrimBox[9.54 6.12 585.68 835.88]
/BleedBox[9.54 6.12 585.68 835.88]
/Annots 17 0 R
/ArtBox[9.54 18.36 585.68 842]
/Rotate 90
>>
endobj
If you compare with your former version, you see that
two new content streams have been added, 69 0 at the start and 70 0 at the end;
the resources are not an indirect object anymore but instead are directly included here;
the resources contain a new Font ressource Xi0 at 68 0.
Now let's look at the added objects.
This is the font ressource for Helvetica-Bold named Xi0 at 68 0:
68 0 obj
<</BaseFont/Helvetica-Bold
/Type/Font
/Encoding/WinAnsiEncoding
/Subtype/Type1
>>
endobj
Non-embedded, standard 14 font resources are not complicated at all...
Now there are the additional content streams. iText does compress them, but I'll show them in an uncompressed state here:
69 0 obj
<</Length 1>>stream
q
endstream
endobj
70 0 obj
<</Length 106>>stream
Q
q
0 1 -1 0 595.22 0 cm
q
BT
1 0 0 1 36 540 Tm
/Xi0 12 Tf
0.75 g
(Hello people!)Tj
0 g
ET
Q
Q
endstream
endobj
So the new content stream at the start stores the current graphic state, and the new one at the end retrieves that stored state, changes the coordinate system, positions for text insertion, selects font, font size, and the fill colour, and finally prints a string.