A PDF with different outputs in different PDF viewers (with shades) - pdf

Consider the following PostScript file
[1 0 0.5 0.866 150 550] concat
<<
/ShadingType 2
/Coords [ 0 0 100 100]
/BBox [ 0 0 100 100]
/ColorSpace [ /DeviceRGB ]
/Function
<<
/FunctionType 0
/Domain [0 1]
/Range [0 1 0 1 0 1]
/BitsPerSample 8
/Size [2]
/DataSource <FFA0A0FFE0E0>
>>
/Extend [false false]
>>
shfill
Consider that we convert that file in PDF with GhostScript (ps2pdf) or Adobe Distiller.
The resulting PDF does not render the same way in the different PDF viewers :
In Adobe Reader or Firefox (which uses PDF.js), we have a parallelogram (not a rectangle).
In SumatraPDF (which uses MuPDF) and Chrome (which uses PDFium), we have a rectangle.
Who is right?

In my opinion Adobe Acrobat is right but the specification could be read differently, too.
Your PDF contains the following content stream:
/GS1 gs
q
1 0 .5 .866 150 550 cm
/Sh1 sh
Q
I.e. first the current transformation matrix is changed, it is sheared and squished a bit, and then the shading Sh1 is painted. That shading in turn is defined as
<</BBox[0 0 100 100]/ColorSpace/DeviceRGB/Coords[0 0 100 100]/Function 15 0 R/ShadingType 2>>
I.e. with a 100×100 square bounding box (interpreted as a temporary additional clipping path) and an axial shading along its (0, 0) to (100, 100) diagonal, matching your postscript definition.
The shading operator sh is specified as
Operands
Operator
Description
name
sh
(PDF 1.3) Paint the shape and colour shading described by a shading dictionary, subject to the current clipping path. The current colour in the graphics state is neither used nor altered. The effect is different from that of painting a path using a shading pattern as the current colour. name is the name of a shading dictionary resource in the Shading subdictionary of the current resource dictionary (see 7.8.3, "Resource dictionaries"). All coordinates in the shading dictionary are interpreted relative to the current user space. (By contrast, when a shading dictionary is used in a Type 2 pattern, the coordinates are expressed in pattern space.) All colours are interpreted in the colour space identified by the shading dictionary’s ColorSpace entry (see "Table 77 — Entries common to all shading dictionaries"). The Background entry, if present, is ignored.This operator should be applied only to bounded or geometrically defined shadings. If applied to an unbounded shading, it paints the shading’s gradient fill across the entire clipping region, which may be time-consuming.
(ISO 32000-2:2017, Table 76 — Shading operator)
In particular: All coordinates in the shading dictionary are interpreted relative to the current user space.
Thus, the square bounding box / temporary clip path is squished and sheared by the current transformation matrix to a non-rectangular parallelogram as can be viewed in Adobe Acrobat:
I mentioned above that the specification can be read differently, too: If one considers the BBox entry as the coordinates of two points, the lower left corner and the upper right corner of the box, and applied the transformation before making the result a box, one would get a squished, elongated rectangle as can be viewed in Chrome:
But the BBox here is specified as an array of four numbers giving the left, bottom, right, and top coordinates, respectively, of the shading’s bounding box (ibidem, Table 77 — Entries common to all shading dictionaries) and not as the coordinates of two endpoints of a diagonal. Thus, I'd favor the first interpretation also implemented by Adobe.
I don't have a copy of ISO 32000-2:2020 yet, so maybe this has been clarified one way or the other.
The situation would be different if the shading would have been used in a pattern which would have served as current color during a fill instruction. In that case the specification says:
A pattern’s appearance is described with respect to its own internal coordinate system. Every pattern has a pattern matrix, a transformation matrix that maps the pattern’s internal coordinate system to the default coordinate system of the pattern’s parent content stream (the content stream in which the pattern is defined as a resource). The concatenation of the pattern matrix with that of the parent content stream establishes the pattern coordinate space, within which all graphics objects in the pattern shall be interpreted.
(ISO 32000-2:2017, Section 8.7.2 — General properties of patterns)
In this case the square bounding box with the diagonal axial shading would not have been subject to the current transformation matrix.

Related

How to find the " PDF page origin " for an existing Page...?

Hi I am trying to find the origin i.e x and y coordinates of a page is there any code examples "Using PDFBOX" and also theory that will help to find the origin of the page in the PDF.
By saying that i mean , we need to find wether the origin is
left bottom? right bottom? right top? left top ? or from the middle of the page ?
First of all, I assume we are talking about user space coordinates, not device space coordinates. When rendering a PDF, coordinates eventually are translated to the device space of the rendering target. But device space coordinates are device dependent and, therefore, not really appropriate for generic PDF processing tasks.
The default user space coordinate system of a page
The default user space coordinate system is in particular used for positioning annotations and is the initial user space coordinate system when starting to process the instructions of the page content stream.
This coordinate system is specified by the effective crop box of the page (which defaults to its media box):
The user space coordinate system shall be initialised to a default state for each page of a document. The CropBox entry in the page dictionary shall specify the rectangle of user space corresponding to the visible area of the intended output medium (display window or printed page). The positive x axis extends horizontally to the right and the positive y axis vertically upward, as in standard mathematical practice (subject to alteration by the Rotate entry in the page dictionary).
(ISO 32000-2, section 8.3.2.3 "User space")
Thus, even without considering the page rotation, the origin may be anywhere inside, on the edge, or outside the visible page area, e.g. for the following CropBox values:
[ 0 0 612 792 ] - origin in the lower left
[ 0 -792 612 0 ] - origin in the upper left
[ -306 -396 306 396 ] - origin in the center of the page
[ -1612 1000 -1000 1792 ] - origin off page to the right and below
If you also take page rotation into account, the origin rotates with the page:
Key
Type
Value
Rotate
integer
(Optional; inheritable) The number of degrees by which the page shall be rotated clockwise when displayed or printed. The value shall be a multiple of 90. Default value: 0.
(ISO 32000-2, Table 31 "Entries in a page object")
So e.g. for the crop box [ 0 0 612 792 ] for the following Rotate values:
0 - origin in the lower left
90 - origin in the upper left
180 - origin in the upper right
270 - origin in the lower right
and for the crop box [ -1612 1000 -1000 1792 ]:
0 - origin off page to the right and below
90 - origin off page to the left and below
180 - origin off page to the left and above
270 - origin off page to the right and above
Of course also the directions of the coordinate axis change matching the rotation:
0 - x coordinates increase to the right, y coordinates upwards
90 - x coordinates increase downwards, y coordinates to the right
180 - x coordinates increase to the left, y coordinates downwards
270 - x coordinates increase upwards, y coordinates to the left
The current user space coordinate system of a page
While processing the instructions of a page content stream, the user space may be transformed along, in particular by the cm instruction:
Operands
Operator
Description
a b c d e f
cm
Modify the current transformation matrix (CTM) by concatenating the specified matrix (see 8.3.2, "Coordinate spaces"). Although the operands specify a matrix, they shall be written as six separate numbers, not as an array.
(ISO 32000-2, Table 56 "Graphics state operators")
One use case for this is to have the current coordinate system "the right side up" after rotation.
For example for the crop box [ 0 0 612 792 ] and the page rotation 90, the coordinate system has its origin in the upper left, x coordinates increase downwards, and y coordinates increase to the right. To straighten this out, you'll often find a cm instruction like this at the start of the page content stream:
0 1 -1 0 612 0 cm
After this instruction the origin on the rotated page in our example is again in the lower left, and x coordinates increase to the right and y coordinates upwards.

Images rotated when added to PDF in itext7

I'm using the following extension method I built on top of itext7's com.itextpdf.layout.Document type to apply images to PDF documents in my application:
fun Document.writeImage(imageStream: InputStream, page: Int, x: Float, y: Float, width: Float, height: Float) {
val imageData = ImageDataFactory.create(imageStream.readBytes())
val image = Image(imageData)
val pageHeight = pdfDocument.getPage(page).pageSize.height
image.scaleAbsolute(width, height)
val lowerLeftX = x
val lowerLeftY = pageHeight - y - image.imageScaledHeight
image.setFixedPosition(page, lowerLeftX, lowerLeftY)
add(image)
}
Overall, this works -- but with one exception! I've encountered a subset of documents where the images are placed as if the document origin is rotated 90 degrees. Even though the content of the document is presented properly oriented underneath.
Here is a redacted copy of one of the PDFs I'm experiencing this issue with. I'm wondering if anyone would be able to tell me why itext7 is having difficulties writing to this document, and what I can do to fix it -- or alternatively, if it's a potential bug in the higher level functionality of com.itextpdf.layout in itext7?
Some Additional Notes
I'm aware that drawing on a PDF works via a series of instructions concatenated to the PDF. The code above works on other PDFs we've had issues with in the past, so com.itextpdf.layout.Document does appear to be normalizing the coordinate space prior to drawing. Thus, the issue I describe above seems to be going undetected by itext?
The rotation metadata in the PDF that itext7 reports from a "good" PDF without this issue seems to be the same as the rotation metadata in PDFs like the one I've linked above. This means I can't perform some kind of brute-force fix through detection.
I would love any solution to not require me to flatten the PDF through any form of broad operation.
I can talk only about the document you`ve shared.
It contains 4 pages.
/Rotate property of the first page is 0, for other pages is 270 (defines 90 rotation counterclockwise).
IText indeed tries to normalize the coordinate space for each page.
That`s why when you add an image to pages 2-4 of the document it is rotated on 270 (90 counterclockwise) degrees.
... Even though the content of the document is presented properly oriented underneath.
Content of pages 2-4 looks like
q
0 -612 792 0 0 612 cm
/Im0 Do
Q
This is an image with applied transformation.
0 -612 792 0 0 612 cm represents the composite transformation matrix.
From ISO 32000
A transformation matrix in PDF shall be specified by six numbers,
usually in the form of an array containing six elements. In its most
general form, this array is denoted [a b c d e f]; it can represent
any linear transformation from one coordinate system to another.
We can extract a rotation from that matrix.
How to decompose the matrix you can find there.
https://math.stackexchange.com/questions/237369/given-this-transformation-matrix-how-do-i-decompose-it-into-translation-rotati
The rotation is defined by the next matrix
0 -1
1 0
This is a rotation on -90 (270) degrees.
Important note: in this case positive angle means counterclockwise rotation.
ISO 32000
Rotations shall be produced by [rc rs -rs rc 0 0], where rc = cos(q)
and rs = sin(q) which has the effect of rotating the coordinate system
axes by an angle q counter clockwise.
So the image has been rotated on the same angle in the counter direction comparing to the page.

How do Text Objects in PDF work?

I have a PDF document of which I would like to remove watermarks as automatically as possible to get better results from pdftotext.
After uncompressing it with pdftk I see the watermark almost in plain text:
BT
1 0 0 1 277.40012 755.2005 Tm
0.501961 0.501961 0.501961 rg /R1 gs /R2 8 Tf
[()]TJ
0 0 Td
[(Abc)30(defghi K)30(lm)-40(no)]TJ
-5.423981 -9.600038 Td
[()]TJ
0 0 Td
[(Apr 01, 2017 12:34)]TJ
ET
The watermark is
Abcdefghi Klmno
Apr 01, 2017 12:34
After skimming through Document management — Portable document format (especially page 248f), I found the following:
BT: Begin Text
Tm: Text matrix - what is that?
x y Td: Move to the start of the next line with an offset of (x, y)
TJ: Text showing
Tf: Text state
ET: End Text
What I don't understand is all the numbers and why
[(Abc)30(defghi K)30(lm)-40(no)]TJ
Does it increase the space between Abc and defghi K and decrease the space between lm and no (seems so, looking at Figure 46 on page 259)? By what unit?
What does Tf do?
Could somebody please explain that?
What I don't understand is all the numbers and why
[(Abc)30(defghi K)30(lm)-40(no)]TJ
Does it increase the space between Abc and defghi K and decrease the space between lm and no (seems so, looking at Figure 46 on page 259)?
Nearly so, the positive value decreases and the negative value increases, cf. Table 109 – Text-showing operators in the PDF specification:
array
TJ :
Show one or more text strings, allowing individual glyph positioning. Each element of array shall be either a string or a number. If the element is a string, this operator shall show the string. If it is a number, the operator shall adjust the text position by that amount; that is, it shall translate the text matrix, Tm. The number shall be expressed in thousandths of a unit of text space (see 9.4.4, "Text Space Details"). This amount shall be subtracted from the current horizontal or vertical coordinate, depending on the writing mode. In the default coordinate system, a positive adjustment has the effect of moving the next glyph painted either to the left or down by the given amount. Figure 46 shows an example of the effect of passing offsets to TJ.
The figure is misleading, obviously some type-setting program scrambled up the effect the author wanted to show. The actual source of the figure looks like this:
BT
/T1_2 1 Tf
0 Tc 8.7503 0 0 8.7503 118.989 450.2115 Tm
[([ \()11(A)53(W)57(A)79(Y again\) ] )41(T)43(J)]TJ
40.0016 0 0 40.0015 296.9949 440.2111 Tm
[(A)53(W)57(A)79(Y again)]TJ
8.7503 0 0 8.7503 118.989 403.2097 Tm
[([ \()11(A)9(\) 120 \()-50(W)-55(\) 120 \()11(A)9(\) 95 \()-41(Y again\) ] )41(T)43(J)]TJ
40.0016 0 0 40.0015 296.9949 392.2093 Tm
(AWAY again)Tj
ET
By what unit?
thousandths of a unit of text space, cf. the quote above.
Text space is the coordinate system in which text is shown. It shall be defined by the text matrix, Tm, and the text state parameters Tfs, Th, and Trise, which together shall determine the transformation from text space to user space.
This often coincides with a single unit in glyph space
What does Tf do?
According to Table 105 – Text state operators in the PDF specification
font size Tf :
Set the text font, Tf, to font and the text font size, Tfs, to size. font shall be the name of a font resource in the Font subdictionary of the current resource dictionary; size shall be a number representing a scale factor. There is no initial value for either font or size; they shall be specified explicitly by using Tf before any text is shown.
The only thing I don't understand now is the line
0.501961 0.501961 0.501961 rg /R1 gs /R2 8 Tf
Can you explain that, too?
The instruction
0.501961 0.501961 0.501961 rg
sets the fill color to a medium gray in an RGB color space.
Then
/R1 gs
sets additional graphics state parameters from the ExtGState resource named R1; probably here some transparency effect is defined.
Finally
/R2 8 Tf
sets the font to one defined by the Font resource named R2 and the font size to 8.
Partial answer
Tf
font size Tf
sets the font and the size (see page 244)
gs
dictName gs sets the graphic state:
(PDF 1.2) Set the specified parameters in the graphics state.
dictName shall be the name of a graphics state parameter
dictionary in the ExtGState subdictionary of the current resource
dictionary (see the next sub-clause).
I am not too sure what \R1 means.
rg
1.0 1.0 0.0 rg % Set nonstroking colour to yellow
Hence 0.501961 0.501961 0.501961 rg sets the color to some gray value.
Text matrix
The Text matrix is an affine transformation matrix as explained in this answer.
Hence
1 0 0 1 0 0 Tf
doesn't change anything.
The matrix 1 0 0 1 277.40012 755.2005 Tm moves the text right (?) by 277.40012 text units (?) and down by 755.2005 text units.

Clipping path seems to be outside of text

recently I wanted to construct a PDF document which should have text clipping: With 4 Tr I tried to define the text as clipping area. But when I wanted to fill the lower part of the text with red color, the result was reversed.
Do anyone knows, why?
Thanks for any answer!
stream
BT
4 8 Td
0.8 0.2 0.7 rg % Writing lila.
4 Tr % Fill & Use text as clipping area.
/TR 32 Tf
(Hallo Welt) Tj
1 0 0 rg % Fill in red.
0 0 200 20 re F % <- Mistake?
ET
What I wanted to have:
What I got:
Have a look at the specification ISO 32000-1:
The behaviour of the clipping modes requires further explanation. Glyph outlines shall begin accumulating if a BT operator is executed while the text rendering mode is set to a clipping mode or if it is set to a clipping mode within a text object. Glyphs shall accumulate until the text object is ended by an ET operator; the text rendering mode shall not be changed back to a nonclipping mode before that point.
(section 9.3.6 Text Rendering Mode )
In your sample you don't wait until the ET for the clipping path to take effect. So, when you are painting the red rectangle, your special clipping path is not yet in effect.
Furthermore your operation sequence actually is invalid! Neither path construction nor path painting operators (i.e. neither your 0 0 200 20 re nor your F) are allowed inside a text object, cf. Figure 9 – Graphics Objects in the specification:
Thus, strictly speaking your PDF viewer had better refuse to draw your content stream at all.

PDF transformation matrix has a scaling of 50 units

I'm trying to highlight some text with a glyph width of 1000 (which corresponds to 1 unit of text space)and font size of 1; the transformation matrix is [50 0 0 50 0 0]. The result is text that is too big. But this is not the case. The text that is being displayed is not big at all; it's a normal size.
Any PDF reader I open the file with has no problems highlighting the word, which means that I'm missing something somewhere.
Currently I'm checking for the default font and the font array in the fonts dictionary, the font size, and the transformation matrix. Is there any other way to scale text in a PDF besides the ones I just mentioned?
This answer combines the comments to the original question:
Currently I'm checking for the default font and the font array in the fonts dictionary, the font size, and the transformation matrix. Is there any other way to scale text in a PDF besides the ones I just mentioned?
A few possibility coming to my mind immediately:
A new transformation matrix (argument to cm) does not replace the old one; instead it is multiplied to it (from the left).
In case of q ... Q you have to consider resets of the current transformation matrix.
(The current transformation matrix, line widths, colors, overprint settings, and much, much more are part of the graphics state. To get an impression, have a look at the entries in tables 57 and 58 of the PDF specification ISO 32000-1. At least all the properties described there are part of the graphics state and, therefore, saved during q and restored during Q.)
Furthermore there is the text matrix to consider.
Finally the UserUnit entry of the page might change the rules.
So there's more to look at than the text positioning operators.
For a good overview have a look at section 9.4.4 Text Space Details of the PDF specification, especially Note 2 therein. (Thanks to #plinth.)