I made a 3x3 table that all cell is stroked but the outer lines are thick by Excel. Then exported to PDF and got the following stream (decoded from Flate).
q
49.56 728.5 169.25 60.12 re
W* n
/P <</MCID 0>> BDC /GS6 gs
0 g
49.56 729.58 2.88 59.04 re
f*
0.14 w
/GS7 gs
0 G
2 J 1 j
105.68 785.68 m
105.68 732.52 l
S
105.62 732.46 0.96 53.28 re
f*
I don't believe that this stream is complete to draw the table. Is something missed or a kind of interpolation working? Of course table is rendered properly on Acrobat Reader.
The image is just to explain and is not what I made so its size differs.
Update
Please refer to the actual file:
https://drive.google.com/file/d/1We2jri-Y04fBYJcZ96HIs05DPeTSDOIc/view?usp=sharing
Actually the content stream is a bit longer:
q
49.56 728.5 169.25 60.12 re
W*
n
/P <</MCID 0 >> BDC
/GS6 gs
0 g
49.56 729.58 2.88 59.04 re
f*
0.14 w
/GS7 gs
0 G
2 J
1 j
105.68 785.68 m
105.68 732.52 l
S
105.62 732.46 0.96 53.28 re
f*
160.76 785.68 m
160.76 732.52 l
S
160.7 732.46 0.96001 53.28 re
f*
214.85 729.58 2.88 56.16 re
f*
52.44 785.74 165.29 2.88 re
f*
52.5 768.76 m
214.79 768.76 l
S
52.44 767.86 162.41 0.96 re
f*
52.5 750.28 m
214.79 750.28 l
S
52.44 749.38 162.41 0.96 re
f*
52.44 729.58 165.29 2.88 re
f*
Q
EMC
All lines are drawn using filled narrow rectangles. The inner lines additionally are drawn as stroked single-edge paths. Those latter stroked paths are not necessary for the appearance.
You probably should check the code for FLATE decoding the content stream which returned you only a partial result.
Related
I have to write a multi lingual text a pdf using C++. I have unicode values as well as glyph id values with their advances and displacements for the string input.
But I need to know how to position the dependent glyph with the independent base glyph.
Suppose if I have a advance and displacement values using FreeType / HarfBuzz, how should I input these values into the pdf content stream along with the glyph ids in the input.
I have tried the output values of FreeType & HarfBuzz, which could print the individual glyphs properly, but the positioning of the glyphs with its base glyph is not proper still, even if i used the advance and displacement values given in their outputs.
I just need the logic of how to use the output values in the content stream to deliver a proper readable word/letter.
Example:
Text = tamil letter + hindi letter.
I need to print this output.proper output
But currently only I am able to print this. improper output
Tamil combined letter:
வ = U+0BB5 TAMIL LETTER VA = base glyph
ா = U+0BBE TAMIL VOWEL SIGN AA = dependent glyph
HarfBuzz run:
hb-shape.exe -O json -u u+0bb5,u+0bbe --no-glyph-names "C:\\Windows\\Fonts\\Nirmala.ttf"
gid output:
[{"g":2953,"cl":0,"dx":0,"dy":0,"ax":2111,"ay":0},{"g":2959,"cl":0,"dx":0,"dy":0,"ax":1453,"ay":0}]
Hindi combined letter:
म = U+092E DEVANAGARI LETTER MA = base glyph
ि = U+093F DEVANAGARI VOWEL SIGN I = dependent glyph
HarfBuzz run:
hb-shape.exe -O json -u u+092e,u+093f --no-glyph-names "C:\\Windows\\Fonts\\Nirmala.ttf"
gid output:
[{"g":302,"cl":0,"dx":0,"dy":0,"ax":532,"ay":0},{"g":273,"cl":0,"dx":0,"dy":0,"ax":1379,"ay":0}]
Subjecting these output values into the formula,
PDF doc formula
Assuming unity for all variables except width and advance,
by obtaining the width value using FreeType and computing them.
Glyph Advance values for four glyphs in order:
tx = 1769
tx = 1132
tx = 1586
tx = 1448
If I provide these values in the content stream in the order as
<glyph id 1> tx 1 <glyph id 2> tx 2 <glyph id 3> tx 3 <glyph id 4> tx 4
Content stream:
/OC /oc2 BDC q BT /FXF1 1 Tf 70.866142 0.000000 0.000000 70.866142 28.346457 141.732285 Tm[<0B89>-1769<0B8F>-1132<0111>-1586<012E>-1448]TJ ET Q EMC
PDF Doc says (+)ve value of advances will move the text towards left.
Is it other way...?
Or if the difference of the advances is to be obtained...?
Additional PDF objects:
Font descriptor object,Base font object,Font object.
I have tried using only advance values and only computed values also.
The only problem is the horizontal & vertical space within combined glyphs, which also affects the spacing between subsequent glyphs.
Any of these does not render the glyphs as legible, atleast in a generalised programmatic manner.
From my analysis of #mkl at various stack overflow places, I suspect the need for individual transformation matrix or Td for each glyph. But is it that complex...?
As per my thought, it must be easily be rendered.
If individual transformation matrix or Td is the need, then how to compute the values to be supplied in for them.
Any help & guidance is welcome and much appreciated.
Thank you.
It helps to work out pdf as plain text you can compile by save in notepad.
Here I am altering a batch.cmd (work in progress :-) to test my compiler handles the changes as text but you can use raw pdf in editor too. beware cut and paste may need a value or two changed Also unknown yet how you can easily reference non Latin fonts (next hurdle after images, which are almost done), so I used "symbol" font as illustrative of those positioning mods.
Note for specific queries #mkl is the expert I simply do programming by examples, that function not by the book.
%PDF-1.0
%µ¶µ¶
1 0 obj<</Type/Catalog/Pages 2 0 R>>endobj
2 0 obj<</Type/Pages/Count 1/Kids[3 0 R]>>endobj
3 0 obj<</Type/Page/Parent 2 0 R/MediaBox [0 0 594 792]/Resources<</Font<< /F1 4 0 R /F2 5 0 R>>>>/Contents 6 0 R>>endobj
4 0 obj<</Type/Font/Subtype/Type1/BaseFont/Helvetica>>endobj
5 0 obj<</Type/Font/Subtype/Type1/BaseFont/Symbol>>endobj
%Comment the following /Length 0999 is a dummy value it should be altered to equal decimal stream length, but most readers will ignore or work around invalid
6 0 obj<</Length 1326>>
stream
q
BT /F1 20 Tf 072 740 Td (20 units (default units usually = pts) high Headline) Tj ET
BT /F1 16 Tf 036 700 Td (All text is "Body" text. (no heads or tails)) Tj ET
BT /F1 10 Tf 004 780 Td (Text can be any order see "Body" text above. (Printed by Filename="C:\Users\K\Downloads\Programming\CMDaPDF\MAKE2PDF.cmd") spot the escape errors) Tj ET
BT /F1 12 Tf 036 675 Td (Here # 12 units high you must include just enough text for parts of a line. PDF has no page feeds no wrapping,) Tj 0 -20 Td (nor \\new line feed, no ¶aragraphs) Tj 86 -15 Td (nor carriage \r\\return. \n\r ) Tj 100 5 Td ( It is not \007\010\011\012\\tabular, each page is one row of multiple pages,) Tj 50 -15 Td (each page is one text column wide .[ ×] no yes check) Tj 0 -10 Td (each row is one text column wide .[x] no is yes) Tj 0 -10 Td (each row is one text column wide . · bullet point OK) Tj ET
BT +0.50 Tc -1.4 Tw 999 TL /F1 1 Tf 15 001 10. 30 200.000 440.000 Tm [(Jane A)600(usten)] TJ ET
BT +0.50 Tc 0.00 Tw 000 TL /F2 1 Tf 15 000 000 15 200.000 430.000 Tm [(Ja)-1000(ne Austen)] TJ ET
BT -1.20 Tc 0.00 Tw 999 TL /F2 1 Tf 15 000 000 15 200.000 420.000 Tm [(J)-1200(a)800(ne Austen)] TJ ET
BT +0.00 Tc 0.00 Tw 000 TL /F2 1 Tf 15 000 000 15 200.000 410.000 Tm [(Jane A)100(us)-500(ten)] TJ ET
Q
endstream
xref
0 7
0000000000 65535 f
0000000019 00000 n
0000000065 00000 n
0000000117 00000 n
0000000242 00000 n
0000000306 00000 n
0000000527 00000 n
trailer<</Size 7/Root 1 0 R>>
startxref
1903
%%EOF
I'm trying to write a justified paragraph in a PDF with multiple formats (sizes, italics, bold, colors) and to achieve this I could use the Graphics State Stack to avoid repeating text operators, but it seems that the behavior of the Graphics State Stack depends on the PDF reader. Do I have to repeat the text operators every time I want to change text format? or there is a better way to achieve what I need?
I have the following PDF stream to test the Graphics State Stack of PDF:
BT
1 0 0 1 56.69 785.19 Tm
0 -12.1 Td
/F1 11 Tf
1.79 Tw
(rzo motáúe issjstózñ x vasreqyxñ ómfzzííh nohé hábúgíoújé úyz túit k wf ñxaóúgsri rcémwewá)Tj
0 -16.5 Td
5.1 Tw
(óaxhkd óáfythra)Tj
q
/F2bi 10 Tf
0.835 0.283 0.833 rg
4.1718 Tw
(olvéd cjtwymelgv stzr cc uxnugtqúic)Tj
q
/F3b 15 Tf
0.491 0.895 0.74 rg
15 Tw
(q hwvúñóál íu vpfíxht)Tj
0 -16.5 Td
(qfyébávrx vkámday)Tj
Q
Q
(cúprnfr úhwñ rá wdwñ óyxáumvpn nmrdó)Tj ET
In the Ubuntu PDF reader the q operator doesn't affect the Td operator.
In the Chrome PDF reader the q operator do affect the Td operator.
The save and restore graphics state operators are not allowed in text objects, i.e. q and Q between BT and ET is invalid.
Thus, your content stream is invalid and the behavior of pdf viewers attempting to display it is undefined.
I'm experiencing a real difficulty when trying to compute (tx,ty) position of text objects from a parsed PDF stream.
I have a following stream code:
BT
0.75 0.68 0.67 0.902 k
/GS0 gs
/TT0 1 Tf
-0.018 Tc 7.56 0 0 7.56 77.1871 528.3107 Tm
(Text line 1)Tj
-0.019 Tc 0 -1.917 TD
(Text line 2)Tj
-0.017 Tc 0 -2.917 TD
(HEADER)Tj
ET
q
43.167 489.881 7.56 7.56 re
W n
BT
/TT0 1 Tf
0 Tc 7.56 0 0 7.56 43.1671 491.7707 Tm
(INDEX)Tj
ET
When I open this PDF in some PDF reader, the HEADER and INDEX objects appear exactly next to each other, as they were in the same line.
However, when calculating HEADER's ty value from previous Tm (528,3107), I get 491,7657 which is 0,0050 lower than INDEX's ty (491,7707). In other parts of file the more text paragraph has the greater is this difference.
Basically what I do is multiply Tm's scaling factor (7,56) by TD's ty deltas. Obiously, I'am doing it wrong, but still - on the net there is little docs for dummies like myself...
So my question is - how to the other PDF readers interpret HEADER and INDEX ty values as equal, so they print it at the same ty?
I am on ubuntu.
I have a pdf file with pages divided into a grid. Each block of the grid contains name/age/dob/photo of a candidate. some records have a watermark "disqualified"
I need to scrape his pdf, with disqualified candidates in a separate list.
Using pyPdf I was able to get individual records, but it also includes watermarked candidates.
How to detect the watermark? If I can get the coordinates of the watermark, how do I match it with the candidate?
I am open to solutions other than python pyPdf
(Actually this is not an answer but merely an analysis to bit for a comment.)
I don't know pyPdf (or any python PDF classes) myself, but here is how the watermark is created for a sample entry; based upon this, anyone knowing pyPDF well enough, may more easily advice.
The Roundup
Depending on how pyPDF (or other python PDF classes) allows access to the page content, there are two major basic approaches:
If the class returns information on content (text and image) in their order in the page content stream: The watermark image xobject is referred to right before the data of the entry. Thus, any entry preceded by the drawing of a xobject image is marked.
If otherwise the information are not given in the order indicated by the page content stream, coordinate comparison must be used which per se is quite straight forward. In that case it might be of interest that the images are inserted with a [0.1 0 0 0.1 0 0] transformation matrix in action while the text is drawn with an identity transformation matrix.
The Details
This is entry # 200; the other watermarked entry is constructed similarly:
Watermarking is done by means of an image xobject. There is but one image xobject defined for the page used by both watermarked entries:
4 0 obj
<</Type/Page/MediaBox [0 0 595 841]
/Rotate 0/Parent 3 0 R
/Resources<</ProcSet[/PDF /ImageC /ImageI /Text]
/ColorSpace 18 0 R
/ExtGState 19 0 R
/XObject 20 0 R
/Font 21 0 R
>>
/Contents 5 0 R
>>
endobj
20 0 obj
<</R17
17 0 R>>
endobj
17 0 obj
<</Subtype/Image
/ColorSpace 16 0 R
/Width 128
/Height 88
/BitsPerComponent 8
/Filter/FlateDecode/Length 463>>stream
[...]
endstream
endobj
In the content stream this xobject /R17 is inserted right before the data of the entry itself is drawn:
q 0.1 0 0 0.1 0 0 cm
[...]
q 1045 0 0 495 462.5 6510.5 cm
/R17 Do
Q
q
10 0 0 10 0 0 cm BT
0.000487366 Tc
/R10 8 Tf
1 0 0 1 86 650.75 Tm
(Sex : Male)Tj
0.000304794 Tc
-64 0 Td
(Age : 43)Tj
-0.000140686 Tc
-1 11.05 Td
(House No :)Tj
-0.00002085 Tc
1 31.95 Td
(Name :)Tj
0.00008575 Tc
/R12 7.15 Tf
25.5 17.8 Td
( 200 )Tj
ET
Q
1547.5 6475 485 535.5 re
S
q
10 0 0 10 0 0 cm BT
-0.000403137 Tc
/R14 8 Tf
1 0 0 1 145.1 708.5 Tm
(XVX0001081)Tj
0.000421651 Tc
/R14 7.05 Tf
-90.35 -14.95 Td
(Ramesh Kumar)Tj
0.000373332 Tc
/R10 7.05 Tf
-33 -12.75 Td
(Father's )Tj
0.000193787 Tc
7.3 TL
(Name)'
0.00037774 Tc
/R14 7.05 Tf
40.25 1.8 Td
(Ram Singh)Tj
0 Tc
2.5 -11.85 Td
(37)Tj
0.00137196 Tc
/R12 7.15 Tf
-5.25 13.35 Td
(:)Tj
I have a source pdf which I am modifying by adding text objects. I am using "Incremental Updates" which is mentioned in the PDF specification. But while adding text objects using this method I am making some mistakes due to which the pdf doesn't render properly in Adobe Reader 11. When the pdf is opened and I double-click on it, the added text objects get deleted. I figured out that this is due to text annotation.
Now I want to know how a new text object can be added using incremental update? How do the Contents and RC of a free text annotation have to be to maintained?
Also is it possible to disable or delete the annotation so that my problem can be avoided easily? Because I want a simple pdf, I don't want annotation options.
The source pdf I am using is here.
The modified pdf after adding text object is here.
I am not sure that source pdf is itself proper according to pdf specification.
First off let me show you how easy things are if you can use a decent PDF library. I use iTextSharp as an example but the same can also be done with others like PDFBox or PDFNet (already mentioned by #Ika in his answer):
PdfReader reader = new PdfReader(sourcePdf);
using (PdfStamper stamper = new PdfStamper(reader, targetPdfStream)) {
Font FONT = new Font(Font.FontFamily.HELVETICA, 12, Font.BOLD, new GrayColor(0.75f));
PdfContentByte canvas = stamper.GetOverContent(1);
ColumnText.ShowTextAligned(
canvas,
Element.ALIGN_LEFT,
new Phrase("Hello people!", FONT),
36, 540, 0
);
}
(Derived from the Webified iTextSharp Example StampText.cs explained in chapter 6 of iText in Action — 2nd Edition.)
(Which PDF library you choose, depends on your general requirements and available license models.)
If, in spite of the ease of use of such PDF libraries, you insist on doing it manually, here some remarks:
First you have to find the Page dictionary of the page you want to add content to. Depending on the type of PDF this already might require decompression of object streams etc. but in your sample modified1.pdf that is not necessary:
7 0 obj
<</Rotate 90
/Type /Page
/TrimBox [ 9.54 6.12 585.68 835.88 ]
/Resources 8 0 R
/CropBox [ 0 0 595.22 842 ]
/ArtBox [ 9.54 18.36 585.68 842 ]
/Contents [ 9 0 R 10 0 R 11 0 R 12 0 R 13 0 R 14 0 R 15 0 R 16 0 R ]
/Parent 6 0 R
/MediaBox [ 0 0 595.22 842 ]
/Annots 17 0 R
/BleedBox [ 9.54 6.12 585.68 835.88 ]
>>
endobj
You see the array of references to content streams. This is where you have to add new page content to. You can manipulate an existing stream or create a new stream and add it to that array.
(Most PDFs have their content stream compressed. For the general case, therefore, you'd have to decompress a stream before you can work on it. Thus, in my eyes, the easier way would be to start a new stream.)
You chose to manipulate the last referenced stream 16 0 which in your PDF is uncompressed:
16 0 obj
<</Length 37 0 R>>
stream
S 1 0 0 1 13.183 0 cm 0 0 m
[...]
0 10 -10 -0 506.238 342.629 Tm
.13333 .11765 .12157 scn
-.0002 Tc
.0006 Tw
(the Bank and branch on which cheque is drawn\).)Tj
/F1 2 Tf
-15.1279 10.9462 Td
(abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789~!##$%^&*aaaaaaaaaaaaa)Tj
/F2 1 Tf
015.1279 01.9462 Td
(ANAabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789)Tj
ET
endstream
endobj
Your additions, I gather, are the two 3-liners at the bottom which first select a font, then position the insertion point and finally print a selection of letters.
Now you say you added text abc..z and ABC...Z just for testing. But letters b j k q v etc not appearing in the pdf. The problem becomes even more visible for your second addition of letters; here only the capital 'A' and 'N' are displayed.
This is due to the fact that the fonts in question are embedded into the PDF --- fonts are embedded into PDFs to allow PDF viewers on systems which don't have the font in question, to display the PDF --- but they are not completely embedded, only the subset of characters required from that font.
Let's look for the font F2 for which only 'N' and 'A' appear:
According to the page object, the page resources can be found in object 8 0:
8 0 obj
<</Font <</F1 45 0 R /TT2 46 0 R /F2 47 0 R>>
/ExtGState <</GS2 48 0 R>>
/ProcSet [ /PDF /Text ]
/ColorSpace <</Cs6 49 0 R>>
>>
endobj
So F2 is defined in 47 0:
47 0 obj
<</Subtype /Type1
/Type /Font
/Widths [ 722 250 250 250 250 250 250 250 250 250 250 250 250 722 ]
/Encoding 52 0 R
/FirstChar 65
/FontDescriptor 53 0 R
/ToUnicode 54 0 R
/BaseFont /ILBPOB+TimesNewRomanPSMT-Bold
/LastChar 78
>>
endobj
In the referenced ToUnicode map 54 0 you see
54 0 obj
<</Length 55 0 R>>stream
/CIDInit /ProcSet findresource begin 12 dict begin begincmap /CIDSystemInfo <<
/Registry (AAAAAA+F2+0) /Ordering (T1UV) /Supplement 0 >> def
/CMapName /AAAAAA+F2+0 def
/CMapType 2 def
1 begincodespacerange <41> <4e> endcodespacerange
2 beginbfchar
<41> <0041>
<4e> <004E>
endbfchar
endcmap CMapName currentdict /CMap defineresource pop end end
endstream
endobj
In this mapping you see that only character codes 0x41 'A' and 0x4e 'N' are mapped
In your document the font is used only to print "NA" in the amount table cells and for nothing else. Thus, only those two letters 'N' and 'A' are embedded, which results in your addition with that font only outputting these letters.
Thus, to successfully add text to the page you either have to check the font ressources associated with the page for the glyphs they provide (and restrict your additions to those glyphs) or you have to add your own font resource.
As the presence of characters in the encoding often is not as easy to see as it is here (ToUnicode is optional), I would propose, you add your own font ressources. The PDF specification ISO 32000-1 explains how to do that.
Furthermore you state the x and y axis position for the text is not properly displaying in pdf. While you don't say what exactly you mean, you should be aware that in the content stream you can apply affine transformations to the coordinate system of the page, i.e. stretch, skew, rotate, and move the axis.
If you want to use the original coordinate system and not depend on the coordinates to be proper at your additions, you should add an initial content stream to the page containing a q operator (to save the current graphics state on the graphics state stack) and start your additions in a new final content stream with a Q operator (to restore the graphics state by removing the most recently saved state from the stack and making it the current state).
EDIT As a sample I applied the Java equivalent of the C# code at the top to your modified1.pdf with append mode activated. The following objects were changed or added as a result:
The page object 7 0 has been updated:
7 0 obj
<</CropBox[0 0 595.22 842]
/Parent 6 0 R
/Contents[69 0 R 9 0 R 10 0 R 11 0 R 12 0 R 13 0 R 14 0 R 15 0 R 16 0 R 70 0 R]
/Type/Page
/Resources<<
/ExtGState<</GS2 48 0 R>>
/ProcSet [/PDF /Text /ImageB /ImageC /ImageI]
/ColorSpace<</Cs6 49 0 R>>
/Font<</F1 45 0 R/F2 47 0 R/TT2 46 0 R/Xi0 68 0 R>>
>>
/MediaBox[0 0 595.22 842]
/TrimBox[9.54 6.12 585.68 835.88]
/BleedBox[9.54 6.12 585.68 835.88]
/Annots 17 0 R
/ArtBox[9.54 18.36 585.68 842]
/Rotate 90
>>
endobj
If you compare with your former version, you see that
two new content streams have been added, 69 0 at the start and 70 0 at the end;
the resources are not an indirect object anymore but instead are directly included here;
the resources contain a new Font ressource Xi0 at 68 0.
Now let's look at the added objects.
This is the font ressource for Helvetica-Bold named Xi0 at 68 0:
68 0 obj
<</BaseFont/Helvetica-Bold
/Type/Font
/Encoding/WinAnsiEncoding
/Subtype/Type1
>>
endobj
Non-embedded, standard 14 font resources are not complicated at all...
Now there are the additional content streams. iText does compress them, but I'll show them in an uncompressed state here:
69 0 obj
<</Length 1>>stream
q
endstream
endobj
70 0 obj
<</Length 106>>stream
Q
q
0 1 -1 0 595.22 0 cm
q
BT
1 0 0 1 36 540 Tm
/Xi0 12 Tf
0.75 g
(Hello people!)Tj
0 g
ET
Q
Q
endstream
endobj
So the new content stream at the start stores the current graphic state, and the new one at the end retrieves that stored state, changes the coordinate system, positions for text insertion, selects font, font size, and the fill colour, and finally prints a string.