With iText 5.5.4, if I choose to embed CRLs during the signature process, it breaks the PDF/A-2B conformance because of a String too long error :
Adobe Preflight http://img4.hostingpics.net/pics/234201so5.jpg
As you can see in the preflight from Adobe Acrobat Pro 11.0.09, the CRL is 67107 characters long whereas this specific PDF standard require that the maximum length for strings must not exceed 32767 bytes.
Is it a bug in Itext or is there a way to keep PDF/A-2B conformance in that case?
Is it a bug in Itext?
As you can see in the preflight from Adobe Acrobat Pro 11.0.09, the CRL is 67107 characters long whereas this specific PDF standard require that the maximum length for strings must not exceed 32767 bytes
Considering that the problem string is referenced as 1 0 obj/Contents seems to indicate that it actually is the string reserved for the CMS signature container, not the plain CRL.
Looking at the PDF/A specification (I only have part 3 at hands but the parts 2 and 3 should coincide here) there are two sections which seem to contradict each other:
A conforming file shall not contain any string longer than 32767 bytes.
(section 6.1.13 Implementation limits)
and
timestamping and revocation information should be included [in the PDF Signature (a DER-encoded PKCS#7 binary data object)] in order to improve the long-term non-repudiation properties of the signature.
(Annex B Requirements for digital signatures in PDF/A)
In your case following the latter recommendation breaks the former requirement because the PKCS#7 / CMS object is embedded as a string in the PDF.
But as we have a requirement against a recommendation, the recommendation need only be followed as long as the requirements are not violated. The specification even explicitly says so:
When generating signature appearances and any other PDF objects as part of the signing process, a
conforming reader shall ensure that it does not invalidate compliance with this part of ISO 19005
(section 6.4.3 Digital signatures)
Thus indeed, iText signing should fail during the signing process... if you have used the PDF/A aware API, that is. Unfortunately you have not provided your source code, so it is not clear whether or not you have used the PdfAStamper or merely the PdfStamper.
Or is there a way to keep PDF/A-2B conformance in that case?
You didn't supply your source code, so this can only be answered in a more general way.
First of all check whether the space reserved for the signature container is really required or whether there are many padding bytes. In the latter case fine tune the reservation code.
If that does not suffice, embedding the CRL into the signature container automatically results in a violation of PDF/A conformance and, therefore, is not allowed.
One option would then be to try to use OCSP responses instead of CRLs here. OCSP responses usually contain information for only one or at most very few certificates while CRLs can become quite big.
If the CA does not have OCSP servers, I don't see a true PDF/A-2'ish solution. You might want to try to embed validation related information in PAdES part 4 (ETSI TS 102 778-4) DSS (Document Security Store) structures instead of in the signature container. Plain PDF/A viewers most likely won't recognize, let alone use, the CRL there but more versatile viewers, e.g. current Adobe Reader versions, should.
Related
Preamble: I am not trying to merge different PDFs.
I was wondering. Does the signature of a PDF modify the signed portion of the document or is it appended to some other section of the file?
If the signed portion of the document does not include the signatures, it should be possible to transfer the signature of a document to another file containing the exact same document:
In practice, sending identical PDF to person A and B or signing, I get PDF[A] and PDF[B] back, I can then transfer signature of B to PDF[A], getting PDF[A, B].
Is that theoretically possible? Has someone already tried it?
First of all, I assume you mean a special case of electronic signatures which often are referred to as digital signatures: These signatures allow verification using the hash of the bytes signed by the signature.
Furthermore, I assume you mean the interoperable types of digital PDF signatures as specified in the PDF specifications and related documents with the extra condition that the signed bytes encompass the whole signed revision except only the placeholder for the signature container. (The PDF specification allows to sign less but common validators forbid that.)
Under these assumptions:
No, what you want to do is not possible.
Signing does not merely append a signature container somewhere; instead it first extends the PDF with some extra objects to represent the signature and hold its signature container, and then it creates a signature for that prepared PDF, not for the original one.
The preparations by two different persons A and B most likely are not identical, so the ranges signed by them differ.
Furthermore, real "parallel signing" of the identical content is not possible in interoperable digital PDF signing, only a consecutive, serial signing. Thus, if you have a PDF with multiple signatures, the bytes signed by the second one actually include the first signature:
Thus, you cannot simply transfer the signature of B as a second signature to the PDF already signed by A because a second signature has to sign something completely different than a first one.
(That being said, there is a larger signing software house whose software used to create signed PDFs with multiple SignerInfos in a single signature container; this is forbidden in the PDF specification for interoperable signatures but a situation validators seldom check for. During validation Adobe Acrobat here only validated the first SignerInfo; some other software only validated the last; in the end this only caused a lot of confusion.)
I'm generating a PDF file with puppeteer. It automatically generates a PDF-1.4 file. Then, I use dss to digitally sign it with a PAdES signature. The resulting file can be opened in PDF viewers and PDFStudio seems to correctly parse the document signature.
Is this valid however?
Wikipedia states that the PDF/A-2 (which is based on 1.7) added support for PAdES.
Do I need to generate at least PDF-1.7 (or PDF/A-2) to have a valid PDF file with a valid signature?
Note: I use the term valid in both the technical and legal terms.
The PAdES norm ETSI EN 319 142 characterizes
PAdES signatures profiled in the present document build on PDF signatures specified in ISO 32000-1 with an alternative signature encoding to support digital signature formats equivalent to the signature format CAdES.
The PDF norm ISO 32000-1 characterizes
ISO 32000 specifies a digital form for representing documents called the Portable Document Format or usually referred to as PDF. PDF was developed and specified by Adobe Systems Incorporated beginning in 1993 and continuing until 2007 when this ISO standard was prepared. The Adobe Systems version PDF 1.7 is the basis for this ISO 32000 edition. The specifications for PDF are backward inclusive, meaning that PDF 1.7 includes all of the functionality previously documented in the Adobe PDF Specifications for versions 1.0 through 1.6. It should be noted that where Adobe removed certain features of PDF from their standard, they too are not contained herein.
(This may sound a bit confusing, on one hand PDF 1.7 includes all of the functionality previously documented in the Adobe PDF Specifications for versions 1.0 through 1.6, on the other hand Adobe removed certain features of PDF from their standard. Indeed, some features were removed but I don't believe your PDF 1.4 files are affected.)
Thus, a PDF file like yours claiming a version 1.4 also is a PDF 1.7 by backward inclusiveness and as such can get signed by PAdES signatures.
Thus, yes, PDF 1.4 files can (technically) validly be signed with PAdES signatures. (Unless, obviously, your files explicitly disallow this.)
(Actually one can also view PAdES signatures as adopted in ISO 32000-2; in this case your PDF 1.4 files by backward inclusiveness are also PDF 2.0 and as such can be signed using PAdES signatures as specified there.)
You also enquire about legal aspects. First of all, I am not a lawyer, so don't consider this formal legal consultation.
To start with, though, you have to make clear in which legal system you want to investigate legal validity.
While PAdES originally has been defined in the context of European Union signature regulations, a number of other countries also adopted PAdES as standard for their preferred PDF signatures.
So: Are you wondering about validity as signatures in the context of EU eIDAS signatures? Are you considering specific regulations of EU member states? Or are you wondering about the situation in other countries outside the EU?
In the EU your PAdES signatures should be generally accepted. Even though there may be some member state special regulations in specific contexts, they should only influence your choice of the PAdES profile you request for your signatures from DSS, they should not render your PDF 1.4 source PDFs unusable for PAdES signing.
I don't know specifics about non-EU legal systems with a PAdES preference. But I indeed would be surprised if any would be bothered by your PDFs being PDF 1.4.
In comments the question arose whether the signed file is also still a valid PDF 1.4 and if not whether the version 1.4 in the file header would be a concern.
Obviously PDF 1.4 does not know the details of the PAdES signature encodings. Fortunately, though, the PDF Reference 1.4 actually does not know any specific signature encoding at all! Thus, no signature encoding is invalid as long as it follows the very few rules present in the PDF 1.4 reference, and PAdES signatures do so.
Furthermore, the PDF 1.4 reference allows
A PDF producer or Acrobat plug-in extension may also add keys to any PDF object that is implemented as a dictionary, except the file trailer dictionary (see Section 3.4.4, “File Trailer”).
Thus, any keys added while applying the PAdES signature which are not defined in PDF 1.4 are harmless.
Thus, the PDF 1.4 files with PAdES signatures added are also still valid PDF 1.4 files. Obviously, though, a plain PDF 1.4 viewer does not know how to validate the PAdES signatures. But as it does not know how to validate any signatures at all, that's of no concern.
I was going through the official PDF spec. I came across a digitally signed PDF here. While I was analyzing its catalog dictionary, I saw this:
The digital signature is in the form of a signature field, which specifies the byte range of the content to which the signature applies. Any content added on top of it, like annotation, notes, etc. should go in as incremental updates, so the validity of the original content should continue to hold true (excluding direct editing of the content, like changing the Sample word to Sample2). However, when I open the file in Nitro, add some highlight or notes to it, save it and open it in Acrobat, it now says that the signature is invalid. Which brings me to my questions:
Why is Acrobat showing it as invalid? The signature field does not enforce prevention from adding incremental updates, why exactly is it invalid?
Why is Acrobat not allowing addition of notes or highlights? Nitro allows it, for example. There is no Perms dictionary which would specify a DocMDP level restriction, so what exactly it is that Adobe is interpreting as a document level lock?
As already explained in my answer to your previous question on this topic, the file you call "the official PDF spec" is everything but. The official PDF specification is ISO 32000-1 (since 2008) and ISO 32000-2 (the 2017 update).
That answer also points out the origin of the P entry in the FieldMDP transform dictionary your sreenshot shows:
It comes from the Lock dictionary of the same signature dictionary and is defined in Adobe supplement to ISO 32000, extension level 3, (which being from Adobe unfortunately indeed references the PDF Reference 1.7 instead of ISO 32000-1):
P number *(Optional; Extension Level 3) The access permissions granted for this document. Valid values follow:
1, no changes to the document are permitted; any change to the document invalidates the signature.
This extension to ISO 32000-1 has been added to the standard ISO 32000-2.
Thus,
Why is Acrobat showing it as invalid? The signature field does not enforce prevention from adding incremental updates, why exactly is it invalid?
Because it does enforce prevention of any change, see above.
Why is Acrobat not allowing addition of notes or highlights? Nitro allows it, for example. There is no Perms dictionary which would specify a DocMDP level restriction, so what exactly it is that Adobe is interpreting as a document level lock?
Because Nitro (at least the version you tested) does probably merely support ISO 32000-1 but not Adobe's extension 3 to it let alone ISO 32000-2.
PDF/A-2
PDF/A-2 carries over provisions from the ETSI/PadES standard. (source)
Is PAdES mandatory for PDF/A-2? Or are other signature formats in PDF/A-2 also allowed?
PDF 2.0
ETSI will feed these European-specific elements back into ISO for
inclusion in the next release of the PDF standard, ISO 32000-2. (source)
Will PAdES be mandatory in the new PDF 2.0 standard? Or will PDF 2.0 be compatible to PAdES and other signature formats will be allowed, too?
Is PAdES mandatory for PDF/A-2? Or are other signature formats in PDF/A-2 also allowed?
PAdES is not mandatory. In particular the name "PAdES" is not even mentioned in the PDF/A-2 specification. Merely some requirements have parallels to PAdES requirements, e.g.
When computing the digest for the file, it shall be computed over the entire file, including the signature
dictionary but excluding the PDF Signature itself. This range is then indicated by the ByteRange entry
of the signature dictionary.
This is also a requirement introduced by PAdES signatures, in ISO 32000-1 this merely was a recommendation. De facto, though, this had been made a requirement by Adobe Reader long before.
Although ISO 32000-1 also allows the value of the Contents entry of signature dictionary to be a DER-encoded
PKCS#1 binary data object, that format is not recommended.
This effectively recommends the use of PKCS#7/CMS signature container based PDF signatures. This also parallels PAdES which actually goes a step further and requires it. But naked PKCS#1 object based PDF signatures are not fashionable anyway, not even in the plain PDF world.
ISO 32000-1:2008 allows the inclusion of one or more RFC 3281 attribute certificates to be associated
with the signer certificate. However, a conforming writer should not include them as they are not widely
supported and hence use of this attribute will reduce interoperability.
There is a similarly formulated recommendation in PAdES part 2 (Basic Profile). In PAdES part 3 (PAdES-BES
and PAdES-EPES Profiles) attribute certificates are even forbidden. But nowadays attribute certificates in general are not in fashion anymore.
Thus, there are parallels but that's it.
Will PAdES be mandatory in the new PDF 2.0 standard? Or will PDF 2.0 be compatible to PAdES and other signature formats will be allowed, too?
As PDF 2.0 has not yet been published, this strictly speaking is speculation.
In the last draft I could read, though, PAdES signatures have been added to the existing formats and only adbe.pkcs7.sha1 signatures have been deprecated. As SHA1 (which is used in that format at least once) has forfeited trust, this format should not be used anyways, even in current PDFs.
Thus, neither PDF/A-2 nor ISO 32000-2 enforce PAdES style signatures, adbe.pkcs7.detached style signatures are still valid options. If interoperability and long term validation features are of interest, though, PAdES most likely is the better choice.
I was developing a PDF signing tool. For this I will get a signed sha256 data of PDF in PKCS#1 format and a certificate from a device, I need to use these two to embed inside the PDF. Most of the PDF reader only support PKCS#7 signature.
Is there any way to convert a PKCS#1 signature to PKCS#7 signature (using certificate file) ?
I knew that PKCS#7 = Certificate + Optional raw data + Signature in PKCS#1 format
Your question somewhat oversimplifies the situation.
Concerning PKCS#7 <-> PKCS#1:
Yes, a PKCS#7 signature container contains SignerInfo objects which respectively essentially contain a PKCS#1 style signature and some attributes. BUT this PKCS#1 style signature generally is NOT merely signing the document data but instead is signing a structure of so called "signed attributes". One of these is a document hash, others can be the signing time, a link to the signer certificate, and other information; these extra information is required in many use cases. Only the most primitively built SignerInfo structures sign the document data directly.
In general, therefore, if you simply take a PKCS#1 signature of some data and wrap it in a PKCS#7 container, that signature container won't be accepted.
For details see RFC 3852.
Concerning integrated PDF signatures:
Your description of your implementation-to-be is somewhat vague. You seem to think that the document hash to sign is a hash of the original PDF. In case of integrated PDF signatures this is wrong: In order to create an integrated PDF signature, you first extend the PDF by some data which envelop a placeholder for the PKCS#7 signature container (recommended by the specification) or the PKCS#1 signature to integrate. Then you need to hash this extended PDF except the placeholder. (By the current PDF specification you could hash less than that but this won't be accepted by the current Adobe Acrobat/Reader and shouldn't be accepted by any serious verifier).
For details see ISO 32000-1:2008 as published by Adobe
Depending on the legal requirements you may also have to take into account PDF Advanced Electronic Signatures (PAdES) as specified by ETSI, cf. ETSI standard for EU-compliant electronic signatures. These will become part of ISO 32000-2, aka PDF 2.0.
So, are you still sure your use case allows for those very simple PKCS#7 signatures you have in mind, and are your PKCS#1 source signatures created for the right document? In that case building those containers is easily done by looking at RFC 3852.
Anyways, you should have a look at the whitepaper Digital Signatures for PDF documents by Bruno Lowagie (iText Software).
Looking at RFC 3852 > 5.4. Message Digest Calculation Process
It is possible to simply convert PKCS#1 to PKCS#7 as long as there are not signed attributes in it. If there are then you are stuck as the PKCS#1 coverts ONLY the hash of the content and PKCS#7 with signed attributes required to construct an intermediary structure and hash of that structure is being signed.