I'm using this library https://github.com/vbuch/node-signpdf to sign a pdf document. After I have signed the document I can see the signature when I open the pdf with Foxit reader but not when I open it with Adobe reader DC. I also tried Adobe reader XI but there I can't see it either.
When I open the document in pdf xchange viewer I get this error: non critical errors detected in the xref table.
Any ideas what the problem could be?
That's the file I signed: https://drive.google.com/file/d/1AZvS4sP2Y3FwW4Deod87Dgxc9I0QZkoc/view?usp=sharing
In your example PDF the name of the signature field consists of 10 bytes, 9 bytes with value 0x00 and one byte with value 0x01. Apparently Adobe Reader does not like that field name.
After some experiments it looks like Adobe Reader does not like a field name starting with a 0x00 byte.
Maybe it contains some code that determines string lengths in a c'ish manner and interprets a 0x00 as end-of-string. A field name with a leading 0x00 byte, therefore, is interpreted as empty string, a field name not accepted by Adobe Reader either.
Thus, please use a signature field name made of (in particular starting with) some meaningful characters. As validators usually display the name of the signature field, this is a good idea anyways.
In terms of lowlevel PDF objects:
The signature field object looks like this:
18 0 obj
<<
/Type /Annot
/Subtype /Widget
/FT /Sig
/Rect [0 0 0 0]
/V 17 0 R
/T ( )
/F 4
/P 1 0 R
>>
endobj
but only like this, the string value of the T entry actually contains the above mentioned nine 0x00 bytes and one 0x01 byte. This is the value that must be changed to a non-empty string not starting with 0x00. I would propose not using bytes < 0x20 at all. Furthermore, the dot, 0x2e, must not be used in the name, it is reserved for separating partial names.
Related
I'm trying to create an open source library for Digital Signing of PDF files.
Bad parameter
I got most of it done, but I have a problem that the signature shows the following error:
Error during signature verification.
Adobe Acrobat error.
Bad parameter.
I tried to find the problem, but until now have not found it.
I have created 2 pdf files that are striped of almost all other data, except the needed info.
Does anyone know where this error might originate from?
I have already tried different online and offline validators, but non of them pointed me in the right direction.
Does anyone know if this error might originate from the certificate and not the pdf struture itself?
Invalid byte range
While creating this post I also tested it on other pdf file too, but got the error:
Error during signature verification.
Unexpected byte range values defining scope of signed data.
Details: The signature byte range is invalid
Note a slice of the pdf describes it as:
...
/SubFilter/adbe.pkcs7.detached
/ByteRange[0 4197 22193 30080 ]
/Contents<30820...
I have multiple times recalculated the ByteRange attribute and even tried changing it by one byte in each direction, but that will always result in Signature processing error..
I don't know what else can be incorrect about the ByteRange. (the added spaces are the same as how Acrobat pads the byterange.)
If anyone might have an idea on what the problem might is, let me know.
Files
Here are my resulting files:
result_bad_param_with_image.pdf (mirror1) (mirror2)
result_bad_param_no_image.pdf (mirror1) (mirror2)
result2_invalid_byte_range_with_image.pdf (mirror1) (mirror2)
result2_invalid_byte_range_no_image.pdf (mirror1) (mirror2)
A signature file (same as the Contents field in a pdf, excepts directly in separate file):
signature.der (mirror1) (mirror2)
The content of the signature.der is printed here: https://pastebin.com/W4EGJ2fX
(using openssl cms -inform DER -in signature.der -cmsout -print command)
(I know the signature is self signed and does not contains a lot of info, but this should not matter for this, I think, this was just to create these examples)
Edit: New links after solving some problems and added some extra files:
result.pdf
signature.der
signed_content.der
There are some errors in your signature and an uncommon structure which in the context of digital signatures may result in rejection by a validator.
Incorrect Signed Hash Value Inside Signature Container
Signing in CMS signature containers with signed attributes makes use of two hash values:
the hash value of the signed byte ranges of the PDF; that value is correct in your example files;
the hash value of the signed attributes in the SignerInfo of the signature container; that value is not correct in your example files.
PS: Looking into the mismatch once again, it turns out that your signed attributes are not DER encoded: The DER encoding in particular sorts the elements of a SET in a specific order, and in your case the attribute order is not the DER order. The specification requires the signed attributes to be DER encoded, though.
PPS: In a comment you argued
I just checked the order of the SET and I can not find anything that is wrong with it. Here is my reasoning, let me know what part is incorrect. The items should be order according to the 'key' which in my case is an Oid.
First of all, this reasoning is flawed: The type in question is a set type (more exactly an ASN.1 SET OF), not some map type; and the DER encoding rule set only knows the ASN.1 base types. Thus, that OID (which just is an arbitrary part of the attribute structure) cannot be the generic ordering key.
And indeed, a quick glance at the specification shows:
11.6 Set-of components
The encodings of the component values of a set-of value shall appear in ascending order, the encodings being compared
as octet strings with the shorter components being padded at their trailing end with 0-octets.
NOTE – The padding octets are for comparison purposes only and do not appear in the encodings.
(ISO/IEC 8825-1 / ITU-T Rec. X.690, section 11 "Restrictions on BER employed by both CER and DER")
Thus, in case of the signed attributes essentially you first DER encode the attribute elements and then sort the resulting byte arrays as described above.
As an aside, this issue of your signature merely causes problems in probably half the validators around. Some validators do not check or DER re-encode the signed attributes, so they get the same hash as you get. Others either check the encoding up front (and, therefore, throw an error because of the issue) or simply re-encode the attributes in DER (and, therefore, get a different hash than you get).
Problematic Extended Key Usage of Signer Certificate
Your signer certificate has an extended key usage value 1.3.6.1.4.1.311.80.1 (Microsoft's OID for Document Encryption) and only that. Adobe validation used to only support certificates with either no extended key usage or one or more of the following:
emailProtection
codeSigning
anyExtendedKeyUsage
1.2.840.113583.1.1.5 (Adobe Authentic Documents Trust)
See Enterprise Toolkit » Digital Signatures Guide for IT » A: Changes Across Releases.
Incorrect Incremental Updates
You sign in an incremental update to the original PDF. This in general is a good idea as it allows to extract the unsigned original document.
But one needs to add the incremental update correctly, and in case of result2_invalid_byte_range_no_image.pdf and result2_invalid_byte_range_with_image.pdf it is done incorrectly: The original revision there is created using cross reference tables but your incremental updates use pure cross reference streams. This is incorrect, you have to continue with the same kind of cross references.
When opening documents with a mix of cross reference tables and pure cross reference streams, Adobe Acrobat internally repairs this which in particular relocates signatures and so makes byte ranges incorrect.
Uncommon Signature Field Structure
You use an uncommon signature field structure in your example PDFs, you separate the widget from the field and only update the field, not the widget, in signing.
While this strictly speaking is ok, I would implement the common structures while making the code work at all, and deviate only thereafter.
PS: In a comment you asked whether I could elaborate on this.
Your signing implementation in a first step adds an incremental update with an empty signature field and a widget as indirectly referenced kid, e.g.:
16 0 obj
<<
/Type /Annot
/F 4
/Subtype /Widget
/BS << /Type /Border /S /S /W 0 >>
/Parent 17 0 R
/P 2 0 R
/Rect [141.75 664.89 276.75 702.39]
/AP << /N 18 0 R >>
/MK << /BC [.1882353 .1882353 .1882353] /BG [1.00 1.00 1.00] /R 0 >>
/DA (/TiRo 0 Tf 0 0 0 rg\r
)
>>
endobj
17 0 obj
<<
/Kids [16 0 R]
/FT /Sig
/T (eyJ1c2VySWQiOiIyNzIifQ==)
>>
In another incremental update you then sign the field with a direct signature value but don't change the widget, e.g.:
17 0 obj
<<
/Kids [16 0 R]
/FT /Sig
/T (eyJ1c2VySWQiOiIyNzIifQ==)
/V << /Type/Sig ... >>
>>
This is uncommon in some ways:
Usually for signatures the option to merge field and widget is used.
Usually for signatures (except for usage right signatures) the signature dictionary is not a direct but an indirect value of the key V.
Usually the appearance of a signature field is updated together with the signature dictionary if there is an appearance at all.
Also, unless other form fill-ins shall happen between adding an empty signature field and signing it, fields usually are added and filled in the same document update.
Thus, more common would be a single incremental update (or even full re-save) containing something like this:
92 0 obj
<<
/AP << /N 94 0 R >>
/DA (/MyriadPro-Regular 0 Tf 0 Tz 0 g)
/F 132
/FT /Sig
/MK <<>>
/P 1 0 R
/Rect [117.575 499.561 515.968 520.938]
/Subtype /Widget
/T (Signature3)
/Type /Annot
/V 93 0 R
>>
endobj
93 0 obj
<<
/ByteRange [ 0 3227714 5751810 2789]
...
>>
As said above, though, your structure strictly speaking is ok, too. But the "Bad parameter" only occurs when validating from the widget in the document or from the signature panel, but it does not occur when validating your signature using the "Validate Signature" button of the "Signature Properties" dialog. Because of that I think it's possible that Adobe is iritated by an uncommon structure.
I'we build PDF, using PDFBox. I've visible signature too. I write some text like that:
...
builderSting.append("Tm\n");
builderSting.append(" /F1 " + fontSize + "\n");
builderSting.append("Tf\n");
builderSting.append("(hello world)");
builderSting.append("Tj\n");
builderSting.append("ET");
...
PDStream stream= ...;
stream.createOutputStream().write(builder.toString().getBytes("ISO-8859-1"));
everything works well. but if I write some unicode characters in builderString, there is "???"s instead of text.
that's sample PDF: link here
QUESTION 1) when I see PDF structure , there is Question-Marks instead of text. Yes. and I dont know how to write with unicode characters?
9 0 obj
<<
/Type /XObject
/Subtype /Form
/BBox [100 50 0 0]
/Matrix [1 0 0 1 0 0]
/Resources <<
/Font 11 0 R
/XObject <<
/img0 12 0 R
>>
/ProcSet [/PDF /Text /ImageB /ImageC /ImageI]
>>
/FormType 1
/Length 13 0 R
>>
stream
q 93.70079 0 0 50 0 0 cm /img0 Do Q
BT
1 0 0 1 93.70079 25 Tm
/F1 2
Tf
(????)Tj
ET
endstream
endobj
I've font with Encoding WinAsciEncoding. can i use another encoding in pdfbox?
PDFont font = PDTrueTypeFont.loadTTF(template, new File("//fontName.ttf"));
font.setFontEncoding(new WinAnsiEncoding());
QUESTION 2) I 've embedded font in PDF. but text is not written with this font (in visible singature Rectangle). Why?
Question 3) when I remove font, text was still there (when the text was in english). what is the default font? /F1 - which is is 1st font?
Question 4) How to calculate width of my text in visible signature ? Any ideas?
QUESTION 1) when I see PDF structure , there is Question-Marks instead of text. Yes. and I dont know how to write with unicode characters?
I assume that with unicode characters you mean characters present in Unicode but not in e.g. Latin-1. (Because the letter 'a' for example does have a Unicode representation, too, but most likely won't cause you trouble.)
You call getBytes("ISO-8859-1") on your StringBuilder result. Your unicode characters most likely are not in ISO 8859-1. Thus, String.getBytes returns the ASCII code for a question mark in their respective place.
If the question was merely how to write to an output stream with unicode characters in Java, the answer would be easy: Choose an encoding which contains all you characters, e.g. UTF-8, which all consumers of your program support, and call String.getBytes for that encoding.
The case at hand is different, though, as you want to serialize those information as a PDF form xobject stream. In this context your whole approach is somewhere along the route from highly questionable to completely wrong:
In PDFs, each font might come along with its own encoding which might be similar to a common encoding, e.g. /WinAnsiEncoding, or completely custom. These encodings, furthermore, in many cases are restricted to one byte per character, but in case of composite fonts they can also be multi-byte-encodings.
As a corollary, not all elements of the stream elements need to be encoded using the same encoding. E.g. the operator names Tm, Tf, and Tj are encoded using their ASCII codes while the characters of a string to be displayed have to be encoded using the respective font's encoding (and may thereafter be yet again hex-encoded if added in sharp brackets <>).
Thus, creating the stream as a string and then converting them to bytes with a single encoding only works if all used fonts use the same encoding (for the actually used code points) which furthermore needs to be ASCII'ish to correctly represent the operators.
Essentially, you should directly construct the stream in some byte buffer and for each inserted element use the appropriate encoding. In case of characters to be displayed, therefore, you have to be aware of the encoding used by the currently selected font.
If you want to do it right, first study the PDF specification ISO 32000-1, especially the sections on general syntax and chapter 9 Text.
QUESTION 2) I've embedded font in PDF. but text is not written with this font (in visible signature Rectangle). Why?
In the resources of the stream xobject in question there is exactly one embedded font associated to the name /F0. In your stream, though, you have /F1 2 Tf, i.e. you select a font /F1 at size 2.
Question 3) when I remove font, text was still there (when the text was in english). what is the default font?
According to the specification, section 9.3.1,
font shall be the name of a font resource in the Font subdictionary of the current
resource dictionary [...]
There is no initial value for either font or size
Most likely, though, PDF viewers for the sake of compatibility with old or broken documents use some default font.
Question 4) How to calculate width of my text in visible signature ? Any ideas?
The widths obviously depends on the metrics of the font used (glyph widths in this case) and the graphics state you set (font size, character spacing, word spacing, current transformation matrix, text transformation matrix, ...).
In your case you hardly do anything in the graphics state and, therefore, only the selected font size from it is of interest. so the more interesting part are the character widths from the font metrics. As long as you use the standard 14 fonts, you find the metrics here. As soon as you start using other, custom fonts, you have to read them from the font definition files yourself.
Ad 1)
Could it be that
stream.createOutputStream().write(builder.toString().getBytes("ISO-8859-1"));
should be
stream.createOutputStream().write(builderString.toString().getBytes("UTF-8"));
The conversion in getBytes to ISO-8859-1 would make some special characters missing in ISO-8859-1 a ?.
In adobe acrobat x i was inserting text objects and when it is opened in adobe reader 10 it was opening properly.but in adobe reader 11 when i click on that pdf file text objects gets deleted.why this happens? How to solve it?
The source pdf file click here
The pdf file which has problem when double clicking on it in adobe reader 11.
click here
In a nutshell:
You try to change the contents of a free text annotation by changing its normal appearance stream.
This is insufficient: A compliant PDF viewer may ignore this entry and provide their own appearances. So it's mere luck that older Adobe Reader versions chose to not ignore your change.
Thus, you also need to change the information a PDF viewer is expected to create their own appearance from, i.e. foremost the rich text value of RC (in the free text annotation dictionary) that shall be used to generate the appearance of the annotation, and also the Contents value which is the Text that shall be displayed for the annotation.
Furthermore there are defects in your PDFs:
the cross reference table in your first attempt result.pdf was broken;
the intent (IT value) of the free text annotation in your source files is spelled incorrectly.
In detail:
Your result.pdf is broken. Different PDF viewers may display broken PDFs differently.
Some details:
It has been created based on your Src.pdf in append mode but additionally the following change in the original revision has been made to its /Pages object:
In the source:
6 0 obj
<</Count 6
/Type /Pages
/Kids [ 7 0 R 8 0 R 9 0 R 10 0 R 11 0 R 12 0 R ]
>>
endobj
In the result:
6 0 obj
<</Count 3
/Type /Pages
/Kids [ 7 0 R 8 0 R 9 0 R 12 0 R 11 0 R 10 0 R ]
>>
endobj
So the order of the last three pages was changed (which is ok) and the /Count was reduced from 6 to 3. This is inconsistent as there still are 6 child objects but according to the PDF specification ISO 32000-1, Count is
The number of leaf nodes (page objects) that are descendants of this node within the page tree.
Furthermore the cross reference stream of the appended revision is broken.
xref
0 1
0000000000 65535 f
24 1
0001465240 00000 n
57 1
0001466075 00000 n
66 1
0001466909 00000 n
73 1
0001467744 00000 n
93 1
0001473484 00000 n
131 1
0001478703 00000 n
The entries are 19 bytes long including their respectively ending single byte newline character According to the spec, though,
Each entry shall be exactly 20 bytes long, including the end-of-line marker.
The format of an in-use entry shall be: nnnnnnnnnn ggggg n eol
where [...] eol shall be a 2-character end-of-line sequence
There may be more errors in the PDF but you may want to start fixing these.
EDIT
Now with the new PDF Pay-in.pdf with a proper cross reference at hand, let's look at it more in-depth.
Adobe Preflight complains about many occurances of:
[...]
An unexpected value is associated with the key
Key: IT
Value: /FreeTextTypewriter
Type: CosName
Formal Representation: Annot.AnnotFreeText
Cos ID: 86
Traversal Path: ->Pages->Kids->[0]->Annots->[13]
[...]
Ok, let's look at that object 86:
86 0 obj
<< /P 8 0 R
/Type /Annot
/CreationDate (D:20130219194939+05'30')
/T (winman)
/NM (0f202782-2274-44b8-9081-af4010be86d4)
/Subj (Typewritten Text)
/M (D:20130219195100+05'30')
/F 4
/Rect [ 53.2308 33.488 552.088 826.019 ]
/DS (font: Helv 12.0pt;font-stretch:Normal; text-align:left; color:#000000 )
/AP <</N 107 0 R >>
/Contents (wwww)
/IT /FreeTextTypewriter
/BS 108 0 R
/Subtype /FreeText
/Rotate 90
/DA (16.25 TL /Cour 12 Tf)
/RC (<?xml version="1.0"?>
<body xmlns="http://www.w3.org/1999/xhtml"
xmlns:xfa="http://www.xfa.org/schema/xfa-data/1.0/"
xfa:APIVersion="Acrobat:10.0.0"
xfa:spec="2.0.2"
style="font-size:12.0pt;text-align:left;color:#000000;font-weight:normal;
font-style:normal;font-family:Helv;font-stretch:normal">
<p dir="ltr">
<span style="line-height:16.3pt;font-family:Helvetica">wwww</span>
</p>
</body>)
>>
endobj
Preflight stated that it is unhappy about the line /IT /FreeTextTypewriter. Looking at the PDF specification again uncovers for annotations with /Subtype /FreeText, i.e. Free Text Annotations specified in section 12.5.6.6:
IT name
(Optional; PDF 1.6) A name describing the intent of the free text annotation (see also the IT entry in Table 170). The following values shall be valid:
FreeText The annotation is intended to function as a plain free-text annotation. A plain free-text annotation is also known as a text box comment.
FreeTextCallout The annotation is intended to function as a callout. The callout is associated with an area on the page through the callout line specified in CL.
FreeTextTypeWriter The annotation is intended to function as a click-to-type or typewriter object and no callout line is drawn.
Default value: FreeText
Thus, your value FreeTextTypewriter is invalid (remember, PDF names are case sensitive!). Therefore, the annotation is (slightly) broken which may already result in all kinds of problems.
But there are other important entries here, too, to understand your issue: All you do in your appended changes is to replace the appearance stream in object 107 (as per /AP <</N 107 0 R >>) of this annotation by a different one. But this annotation contains an RC value, too, which according to the specification is
A rich text string (see 12.7.3.4, “Rich Text Strings”) that shall be used to generate the appearance of the annotation.
Thus, any PDF viewer may regenerate the appearance from that rich text description, especially as the specification in section 12.5.2 says about the content of the AP dictionary
Individual annotation handlers may ignore this entry and provide their own appearances.
Thus, simply replacing the normal appearance stream does not suffice to permanently change the appearance of that annotation, you have to change the appearance dictionary and at least remove any alternative source for the appearance.
Furthermore the entry /Contents (wwww) is not replaced by your appended changes either. So a PDF viewer trying to decide whether to use the appearance stream or not will feel tempted to somehow create a new appearance as your appearance stream in no way represents that value.
Especially when starting to manipulate the free text (e.g. when clicking into the PDF in your case), the PDF viewer knows it eventually will have to create a new appearance anyways, and unless the current appearance is as it would have created it anyway, the viewer may prefer to begin anew starting with an appearance derived from the rich text or even the contents value.
I want to create a PDF file which does not contain angle brackets in it's source.
This apparently implies not using the dictionary data type, as this involves << and >>.
Is it possible to completely avoid angle brackets and still create a PDF file with formatted content?
Can it be done hiding in a stream, using a character encoding technique or with an alternative dictionary notation?
The solution is needed for an obfuscation technique; the bracket-problem cannot be circumvented.
I think this is not possible. Every element in a PDF file is contained in some dictionary: the document catalog (the root dictionary), the page objects, the page content streams, all of them are dictionaries that require the char sequence << >>.
Sample catalog dictionary:
1 0 obj
<<
/Pages 2 0 R
/Type /Catalog
>>
endobj
If you want to use a "instructions sequence only" presentation format, you may try using PostScript instead.
Edit after comments:
Using a stream object with some filter encoding will not solve your problem, since you still need to specify the filter type in the stream dictionary.
Example:
5 0 obj
<</Length 6 0 R /Filter /FlateDecode>>
stream
***illegible characters***
endobj
I am demoing an idea I have been playing around with, and while the Adobe specification says that including PS XObjects is not a good idea, some PDF readers should still support this functionality. Anyways, that is beside the fact. I have been using the Adobe PDF specification and have the following PDF object. This merely uses PostScript to generate a pseudo random value and then print it to the page. Ideally, each time this page is rendered a new value should display:
5 0 obj
<< /Type/XObject
/Subtype/PS
/Length 103
>>
stream
/Times findfont 10 scalefont setfont
/str 32 string def
10 20 moveto
rand str cvs show
endstream
endobj
Each time any PDF viewer I have tested this against reads this object I get errors such as:"Error (741): Missing 'endstream'" And similarly for every token in that stream. I am sure my offsets are correct. And while I know my PDF viewer does support some PS for forms and such, is there anything obviously incorrect. If anyone has a sample PDF I can go from, that would be nice. The form examples that I tested my reader against have not been too helpful. If I run just the PS code from GhostView it works fine. Thanks for any insight.
I've scoured my back collection of PDF files and come up with 2 which contain PS XObjects (this really is deprecated). I can't, unfortunately, share tehm as they are customer data files :-(
However, here is an extract from one of them:
74 0 obj
<<
/Type /XObject
/Subtype /PS
/Filter /FlateDecode
/Length 77 0 R
/Name /Ps1
>>
stream
....endstream
Note 1, there is no EOL between the end of data and the 'endstream' token.
77 0 obj
4480
endobj
The offset of the 0x0A following the 'stream' token is 0xdab15, the offset of the 'e' in endstream is 0xdbc96. That is 4481 bytes. SO it looks to me like the /Length should contain all the bytes after the EOL for the 'stream' token' right up to the last byte before the 'e' in the endstream token.
I think it would be OK to insert a 0x0A after the stream data and before the endstream. That would come down to a whitespace after the stream data before the token, and PDF is supposed to be tolerant of whitespace.
This is consistent with the description of the /Length entry for stream dictionaries in Table 3.4 (p62 of the 1.7 PDF reference):
The number of bytes from the beginning of the line fol-lowing the keyword stream to the last byte just before the keyword endstream. (There may be an additional EOL marker, preceding endstream, that is not included in the count and is not logically part of the stream data.) See “Stream Extent,” above, for further discussion.
I think (if I've counted correctly) that the /Length in your example should be 87, assuming one byte line terminators in the PostScript fragment.