Issues with iTextsharp and pdf manipulation - pdf

I am getting a pdf-document (no password) which is generated from a third party software with javascript and a few editable fields in it. If I load this pdf-document with the pdfReader class the NumberOfPagesProperty is always 1 although the pdf-document has 17 pages. Oddly enough the document has 17 pages if I save the stream afterwards. When I now try to open the document the Acrobat Reader shows an extended feature warning and the fields are not fillable anymore (I haven't flattened the document). Do anyone know about such a problem?
Background Info:
My job is to remove the javascript code, fill out some fields and save the document afterwards.
I am using the iTextsharp version 5.5.3.0.
Unfortunately I can't upload a sample file because there are some confidental data in it.
private byte[] GetDocumentData(string documentName)
{
var document = String.Format("{0}{1}\\{2}.pdf", _component.OutputDirectory, _component.OutputFileName.Replace(".xml", ".pdf"), documentName);
if (File.Exists(document))
{
PdfReader.unethicalreading = true;
using (var originalData = new MemoryStream(File.ReadAllBytes(document)))
{
using (var updatedData = new MemoryStream())
{
var pdfTool = new PdfInserter(originalData, updatedData) {FormFlattening = false};
pdfTool.RemoveJavascript();
pdfTool.Save();
return updatedData.ToArray();
}
}
}
return null;
}
//Old version that wasn't working
public PdfInserter(Stream pdfInputStream, Stream pdfOutputStream)
{
_pdfInputStream = pdfInputStream;
_pdfOutputStream = pdfOutputStream;
_pdfReader = new PdfReader(_pdfInputStream);
_pdfStamper = new PdfStamper(_pdfReader, _pdfOutputStream);
}
//Solution
public PdfInserter(Stream pdfInputStream, Stream pdfOutputStream, char pdfVersion = '\0', bool append = true)
{
_pdfInputStream = pdfInputStream;
_pdfOutputStream = pdfOutputStream;
_pdfReader = new PdfReader(_pdfInputStream);
_pdfStamper = new PdfStamper(_pdfReader, _pdfOutputStream, pdfVersion, append);
}
public void RemoveJavascript()
{
for (int i = 0; i <= _pdfReader.XrefSize; i++)
{
PdfDictionary dictionary = _pdfReader.GetPdfObject(i) as PdfDictionary;
if (dictionary != null)
{
dictionary.Remove(PdfName.AA);
dictionary.Remove(PdfName.JS);
dictionary.Remove(PdfName.JAVASCRIPT);
}
}
}

The extended feature warning is a hint that the original PDF had been signed using a usage rights signature to "Reader-enable" it, i.e. to tell the Adobe Reader to activate some additional features when opening it, and the OP's operation on it has invalidated the signature.
Indeed, he operated using
_pdfStamper = new PdfStamper(_pdfReader, _pdfOutputStream);
which creates a PdfStamper which completely re-generates the document. To not invalidate the signature, though, one has to use append mode as in the OP's fixed code (for char pdfVersion = '\0', bool append = true):
_pdfStamper = new PdfStamper(_pdfReader, _pdfOutputStream, pdfVersion, append);
If I load this pdf-document with the pdfReader class the NumberOfPagesProperty is always 1 although the pdf-document has 17 pages. Oddly enough the document has 17 pages
Quite likely it is a PDF with a XFA form, i.e. the PDF is only a carrier of some XFA data from which Adobe Reader builds those 17 pages. The actual PDF in that case usually only contains one page saying something like "if you see this, your viewer does not support XFA."
For a final verdict, though, one has to inspect the PDF.

Related

iTextSharp v5 - How do you concatenate PDFs in memory?

I am having trouble merging PDFs in-memory. I have 2 memory streams, a master and component stream, the idea is that as each component PDF is built up, the component PDF's bytes are added to the master stream. At the very end of all the components, we have a byte array that's a PDF.
I have the code below, but nothing is copying into my masterStream. I think the issue is with CopyPagesTo, but I'm not familiar enough and the documentation/examples are hard to find.
byte[] updated;
using (MemoryStream masterMemoryStream = new MemoryStream())
{
masterStream.WriteTo(masterMemoryStream);
// Read from master stream (ie. all existing components)
masterMemoryStream.Position = 0;
using (iText.Kernel.Pdf.PdfWriter masterPdfWriter = new iText.Kernel.Pdf.PdfWriter(masterMemoryStream))
using (iText.Kernel.Pdf.PdfDocument masterPdfDocument = new iText.Kernel.Pdf.PdfDocument(masterPdfWriter))
{
using (MemoryStream componentMemoryStream = new MemoryStream())
{
componentStream.WriteTo(componentMemoryStream);
// Read from new component
componentMemoryStream.Position = 0;
using (iText.Kernel.Pdf.PdfReader componentPdfReader = new iText.Kernel.Pdf.PdfReader(componentMemoryStream))
using (iText.Kernel.Pdf.PdfDocument componentPdfDocument = new iText.Kernel.Pdf.PdfDocument(componentPdfReader))
{
// Copy pages from component into master
componentPdfDocument.CopyPagesTo(1, componentPdfDocument.GetNumberOfPages(), masterPdfDocument);
}
}
}
updated = masterMemoryStream.GetBuffer();
}
// Write updates to master stream?
masterStream.SetLength(0);
using (MemoryStream temp = new MemoryStream(updated))
temp.WriteTo(masterStream);
Answer
This is mkl's answer with some of my corrections:
using (MemoryStream temporaryStream = new MemoryStream())
{
masterStream.Position = 0;
componentStream.Position = 0;
using (PdfDocument combinedDocument = new PdfDocument(new PdfReader(masterStream), new PdfWriter(temporaryStream)))
using (PdfDocument componentDocument = new PdfDocument(new PdfReader(componentStream)))
{
componentDocument.CopyPagesTo(1, componentDocument.GetNumberOfPages(), combinedDocument);
}
byte[] temporaryBytes = temporaryStream.ToArray();
masterStream.Position = 0;
masterStream.SetLength(temporaryBytes.Length);
masterStream.Capacity = temporaryBytes.Length;
masterStream.Write(temporaryBytes, 0, temporaryBytes.Length);
}
There are a number of issues in your code. I'll first give you a working version and then go into the issues in your code.
A working version (with an important limitation)
You can combine two PDFs given in MemoryStream instances masterStream and componentStream and get the result in the same MemoryStream instance masterStream as follows:
using (MemoryStream temporaryStream = new MemoryStream())
{
masterStream.Position = 0;
componentStream.Position = 0;
using (PdfDocument combinedDocument = new PdfDocument(new PdfReader(masterStream), new PdfWriter(temporaryStream)))
using (PdfDocument componentDocument = new PdfDocument(new PdfReader(componentStream)))
{
componentDocument.CopyPagesTo(1, componentDocument.GetNumberOfPages(), combinedDocument);
}
byte[] temporaryBytes = temporaryStream.ToArray();
masterStream.Position = 0;
masterStream.Capacity = temporaryBytes.Length;
masterStream.Write(temporaryBytes, 0, temporaryBytes.Length);
masterStream.Position = 0;
}
The limitation is that you have to have instantiated the masterStream with an expandable capacity; the MemoryStream class has a number of constructors only some of which create such an expandable instance while the others create non-resizable instances. For details read here.
Issues in your concept and code
Concatenating PDF files does not result in a valid merged PDF
You describe your concept like this
the idea is that as each component PDF is built up, the component PDF's bytes are added to the master stream
This does not work, though, the PDF format does not allow merging PDFs by simply concatenating them. In particular the (active) objects in a PDF have an identifier number which must be unique in the PDF, concatenating would result in a file with non-unique object identifiers; PDFs contain cross reference structures which map each object identifier to its offset from the file start, concatenating would get all these offsets wrong for the added PDFs; furthermore, a PDF has to have a single root object from which the other objects are referenced directly or indirectly, concatenating would result in multiple root objects.
Writing and immediately overwriting
In your code you have
masterStream.WriteTo(masterMemoryStream);
// Read from master stream (ie. all existing components)
masterMemoryStream.Position = 0;
using (iText.Kernel.Pdf.PdfWriter masterPdfWriter = new iText.Kernel.Pdf.PdfWriter(masterMemoryStream))
Here you write the contents of masterStream to masterMemoryStream, then set the masterMemoryStream position to the start and instantiate a PdfWriter which starts writing there. I.e. your original copy of the masterStream contents get overwritten, surely not what you wanted.
Using MemoryStream.GetBuffer
MemoryStream.GetBuffer does not only return the data written into the MemoryStream by design but the whole buffer; i.e. there may be a lot of trash bytes after the actual PDF in what you retrieve here
updated = masterMemoryStream.GetBuffer();
This may cause PDF processors trying to process your result PDFs to be unable to open the file: PDFs have a pointer to the last cross references at their end, so if you have trash bytes following the actual end of your PDF, PDF processors may not find that pointer.
PS
As worked out in the comments, the code above works fine in case of constantly growing stream lengths (which usually will happen in the use case at hand) but in general one needs to restrict the stream size before writing the new content, e.g. like this:
...
masterStream.Position = 0;
masterStream.SetLength(temporaryBytes.Length); // <<<<
masterStream.Capacity = temporaryBytes.Length;
...

iTextSharp Read Text From Single Layer of PDF

Currently I am using a custom LocationTextExtractionStrategy to extract text from a PDF that returns a TextRenderInfo[]. I would like to be able to determine if a TextRenderInfo object (or PDFString, child of TextRenderInfo) appears in a specific layer. I am not sure if this is possible. To get the layers in a PDF, I am using:
Dictionary<string,PdfLayer> layers;
using (var pdfReader = new PdfReader(src))
{
var newSrc = Path.Combine(["new file location"]);
using (var stream = new FileStream(newSrc, FileMode.Create))
{
PdfStamper stamper = new PdfStamper(pdfReader, stream);
layers = stamper.GetPdfLayers();
stamper.Close();
}
pdfReader.Close();
src = newSrc;
}
To extract the text, I am using:
var textExtractor = new TextExtractionStrategy();
PdfTextExtractor.GetTextFromPage(pdfReader, pdfPageNum,textExtractor);
List<TextRenderInfo> results = textExtractor.Results;
Is there any way that I can check if the individual TextRenderInfo results exist within the layers obtained in the first code snippet. Any help would be much appreciated.
It is possible to get the contents from a single layer, but you'll have to jump through a few hoops to work it out. Specifically, you will have to recreate some of the logic that is provided by the PdfTextExtractor and PdfReaderContentParser.
public static String GetText(PdfReader reader, int pageNumber, int streamNumber) {
var strategy = new LocationTextExtractionStrategy();
var processor = new PdfContentStreamProcessor(strategy);
var resourcesDic = pageDic.GetAsDict(PdfName.RESOURCES);
// assuming you still need to extract the page bytes
byte[] contents = GetContentBytesForPageStream(reader, pageNumber, streamNumber);
processor.ProcessContent(contents, resourcesDic);
return strategy.GetResultantText();
}
public static byte[] GetContentBytesForPageStream(PdfReader reader, int pageNumber, int streamNumber) {
PdfDictionary pageDictionary = reader.GetPageN(pageNum);
PdfObject contentObject = pageDictionary.Get(PdfName.CONTENTS);
if (contentObject == null)
return new byte[0];
byte[] contentBytes = GetContentBytesFromContentObject(contentObject, streamNumber);
return contentBytes;
}
public static byte[] GetContentBytesFromContentObject(PdfObject contentObject, int streamNumber) {
// copy-paste logic from
// ContentByteUtils.GetContentBytesFromContentObject(contentObject);
// but in case PdfObject.ARRAY: only select the streamNumber you require
}
If you're specifically looking to just use PdfTextExtractor or PdfReaderContentParser, and ask the returned TextRenderInfo for the layer it's on, then I'm not sure it will be easily possible. There are a number of problems with that:
TextRenderInfo doesn't store that information, so you'd have to subclass it (which is possible)
you'd have to rewrite the logic that creates the TextRenderInfo objects. This is possible by registering custom IContentOperator objects for all text operators (Tj, TJ, ' and ") with the PdfTextExtractor or PdfReaderContentParser
the hardest part is that you have already lost layer information in ContentByteUtils.GetContentBytesFromContentObject - so you'd need to retain that somehow, which creates its own set of problems.

Merge pdfs with NReco PdfGenerator

In the features section on NReco's site, in the examples list: there is a line about MergePdf.
I have looked in the API-reference and using the intellisense in visualstudio but I can't find anything.
I wan't to merge several pdf's before I sent them in a mail. The Pdf's is generated with nreco wkhtmltopdf with different headers and footers which I could not get to work in the same generate so I splitted the generation and now I want to merge the pdf's again.
Or do I have to get yet another library involved.
Just sharing what I ended up with. At least for now.
It is a modification of the suggested solution with iTextSharp.
public static byte[] MergePdfs(IEnumerable<byte[]> pdfs)
{
using (var memoryStream = new MemoryStream())
{
var document = new Document(PageSize.A4);
var writer = PdfWriter.GetInstance(document, memoryStream);
document.Open();
var writerDirectContent = writer.DirectContent;
foreach (var pdf in pdfs)
{
var pdfReader = new PdfReader(pdf);
var numberOfPages = pdfReader.NumberOfPages;
for (var currentPageNumber = 1; currentPageNumber <= numberOfPages; currentPageNumber++)
{
document.SetPageSize(PageSize.A4);
document.NewPage();
var page = writer.GetImportedPage(pdfReader, currentPageNumber);
writerDirectContent.AddTemplate(page, 1f, 0, 0, 1f, 0, 0);
}
}
document.Close();
return memoryStream.ToArray();
}
}
There are 2 ways how you can achieve the goal you mentioned:
use GeneratePdfFromFiles method which accepts array of WkHtmlInput structures that allow you to specify header/footer separately for each input HTML file. As result one PDF is produced; note that this method requires a valid license key and is not available for a free library users
generate several PDFs in a standard way, and then merge them into one resulting PDF with help of iTextSharp library (free LGPL 4.1.6 can be used for this purpose).

itextsharp split shared xObject streams

I am trying to split shared XObject streams (originally flatten form fields with the same content) in the PDF.
What is the correct way to do this using itextsharp? I am trying the code below but the stream is still shared in the resulting document.
Sample pdf with shared XObject streams flatten.pdf
PdfReader pdf = new PdfReader(path);
PdfStamper stamper = new PdfStamper(pdf, new FileStream("processed.pdf", FileMode.OpenOrCreate, FileAccess.ReadWrite));
EliminateSharedStreams(stamper, 1);
stamper.Close();
virtual public void EliminateSharedXObjectStreams(PdfStamper pdfStamper, int pageNum)
{
PdfReader pdfReader = pdfStamper.Reader;
PdfDictionary page = pdfReader.GetPageN(pageNum);
PdfDictionary resources = page.GetAsDict(PdfName.RESOURCES);
PdfDictionary xObjResources = resources.GetAsDict(PdfName.XOBJECT);
List<PRIndirectReference> newRefs = new List<PRIndirectReference>();
List<PdfName> newNames = new List<PdfName>();
List<PRStream> newStreams = new List<PRStream>();
IntHashtable visited = new IntHashtable();
foreach (PdfName key in xObjResources.Keys)
{
PdfStream xObj = xObjResources.GetAsStream(key);
if (xObj is PRStream && xObj.GetAsName(PdfName.SUBTYPE) != null &&
xObj.GetAsName(PdfName.SUBTYPE).CompareTo(PdfName.FORM) == 0)
{
PRIndirectReference refi = (PRIndirectReference)xObjResources.Get(key);
PRStream xFormStream = (PRStream)xObj;
if (visited.ContainsKey(refi.Number))
{
// need to duplicate
newRefs.Add(refi);
PRStream newStream = new PRStream(xFormStream, null);
newStreams.Add(newStream);
newNames.Add(key);
}
else
visited[xFormStream.ObjNum] = 1;
}
}
if (newStreams.Count == 0)
return;
PdfContentByte canvas = pdfStamper.GetOverContent(pageNum);
PdfWriter writer = pdfStamper.Writer;
for (int k = 0; k < newStreams.Count; ++k)
{
canvas.SaveState();
//add copied stream
PdfIndirectReference newRef = writer.AddToBody(newStreams[k]).IndirectReference;
//change the ref
xObjResources.Put(newNames[k], newRef);
canvas.RestoreState();
}
}
First remarks without a sample document
There are numerous reasons why your code may not work as desired. As you did not supply your sample PDF, I cannot tell which are more relevant and which are not.
You only search xobjects shared on the same page; if a xobject is once used on page one and once on page two, your code cannot identify this.
If you want to be able to find such shares, you'll at least have to use the same IntHashtable instance visited across all calls of EliminateSharedXObjectStreams for the same PdfStamper pdfStamper, e.g. by creating it once outside this method and making it a parameter of your method.
You only check for shared xobjects in the immediate page resources. But form xobjects have their own resources which can contain even more form xobject declarations.
If you want to find such shares, you'll have to recurse into the resources of your page's xobjects, those xobjects' xobjects, etc. pp.
(Strictly speaking you also have to recurse into the form xobjects of patterns and Type 3 Font glyph definitions, but these are unlikely positions to flatten form fields into.)
You only check for shared xobjects with different names. But xobjects can also be shared by referencing the same name multiple times from the same content stream.
If you want to find such shares, you have to analyse the content streams in question to find duplicate usages of the form xobject with the same name.
(By the way, doing so you may also check whether declared xobjects are used at all: if a form xobject is declared in some resources, this does not mean it is used in the context of these resources, it may be an unused resource.)
You don't mark xObjResources (if it itself is indirect) or page (otherwise) as used. If your PdfStamper pdfStamper is working in append mode, your changes may be ignored.
Solution with a sample document
After you provided the information that
It's single page document containing shared streams (xobjects with different names) in the immediate page resources. pdfStamper is not in append mode.
it turned out that the problems mentioned above are not relevant in your case. As you meanwhile also have provided an example document, I could reproduce the issue.
Indeed, your code does not split the shared XObjects. The reason is that the PdfStamper is made for manipulating the PDF in the PdfReader in the state it was in when the stamper was constructed, using stamper methods only. Your code, on the other hand, manipulates objects directly retrieved from the PdfReader after the construction of the stamper. Thus, while your new streams are added to the PDF (actually up front), the changes in the pre-existing XObject resource dictionaries don't make it to the result.
If you want to manipulate objects you retrieve from the reader, you instead should do this before creating the stamper.
This actually should suite you as your code structurally is copied from a Pdfreader method anyways, EliminateSharedStreams, which you adapted to your use case.
The only problem is that that method uses a hidden member variable of the PdfReader class. But you can access that variable bei means of reflection.
Thus, the manipulated method (working on a pure PdfReader) could look like this:
virtual public void EliminateSharedXObjectStreams(PdfReader pdfReader, int pageNum)
{
PdfDictionary page = pdfReader.GetPageN(pageNum);
PdfDictionary resources = page.GetAsDict(PdfName.RESOURCES);
PdfDictionary xObjResources = resources.GetAsDict(PdfName.XOBJECT);
List<PRIndirectReference> newRefs = new List<PRIndirectReference>();
List<PRStream> newStreams = new List<PRStream>();
IntHashtable visited = new IntHashtable();
foreach (PdfName key in xObjResources.Keys)
{
PdfStream xObj = xObjResources.GetAsStream(key);
if (xObj is PRStream && xObj.GetAsName(PdfName.SUBTYPE) != null &&
xObj.GetAsName(PdfName.SUBTYPE).CompareTo(PdfName.FORM) == 0)
{
PRIndirectReference refi = (PRIndirectReference)xObjResources.Get(key);
PRStream xFormStream = (PRStream)xObj;
if (visited.ContainsKey(refi.Number))
{
// need to duplicate
newRefs.Add(refi);
PRStream newStream = new PRStream(xFormStream, null);
newStreams.Add(newStream);
}
else
visited[xFormStream.ObjNum] = 1;
}
}
if (newStreams.Count == 0)
return;
FieldInfo xrefObjField = typeof(PdfReader).GetField("xrefObj", BindingFlags.Instance | BindingFlags.NonPublic);
List<PdfObject> xrefObj = (List<PdfObject>)xrefObjField.GetValue(pdfReader);
for (int k = 0; k < newStreams.Count; ++k)
{
xrefObj.Add(newStreams[k]);
PRIndirectReference refi = newRefs[k];
refi.SetNumber(xrefObj.Count - 1, 0);
}
}
and you can use it like this:
using (PdfReader pdfReader = new PdfReader(sourcePath))
using (Stream pdfStream = new FileStream(targetPath, FileMode.Create, FileAccess.Write))
{
EliminateSharedXObjectStreams(pdfReader, 1);
PdfStamper pdfStamper = new PdfStamper(pdfReader, pdfStream);
pdfStamper.Close();
}
in particular calling EliminateSharedXObjectStreams before constructing the PdfStamper.
If you are after a generic solution, you of course will have to extend the method to remove the restrictions observed in the first part of the answer...
Solution without reflection
The OP found out:
Manipulating PdfReader works as expected. Only thing is that instead of using xrefObj private field, the stream can be add using AddPdfObject:
for (int k = 0; k < newStreams.Count; ++k)
{
PRIndirectReference newRef = pdfReader.AddPdfObject(newStreams[k]);
PRIndirectReference refi = newRefs[k];
refi.SetNumber(newRef.Number, 0);
}
Indeed, this improves the solution substantially.

Existing PDF to PDF/A "conversion"

I am trying to make an existing pdf into pdf/a-1b. I understand that itext cannot convert a pdf to pdf/a in the sense making it pdf/a compliant. But it definitely can flag the document as pdf/a. However, I looked at various examples and I cannot seem to figure out how to do it. The major problem is that
writer.PDFXConformance = PdfWriter.PDFA1B;
does not work anymore. First PDFA1B is not recognized, second, pdfwriter seems to have been rewritten and there is not much information about that.
It seems the only (in itext java version) way is:
PdfAWriter writer = PdfAWriter.getInstance(document, new FileOutputStream(filename), PdfAConformanceLevel.PDF_A_1B);
But that requires a document type, ie. it can be used when creating a pdf from scratch.
Can someone give an example of pdf to pdf/a conversion with the current version of itextsharp?
Thank you.
I can't imagine a valid reason for doing this but apparently you have one.
The conformance settings in iText are intended to be used with a PdfWriter and that object is (generally) only intended to be used with new documents. Since iText was never intended to convert documents to conformance that's just the way it was built.
To do what you want to do you could either just open the original document and update the appropriate tags in the document's dictionary or you could create a new document with the appropriate entries set and then import your old document. The below code shows the latter route, it first creates a regular non-conforming PDF and then creates a second document that says it is conforming even though it may or may not. See the code comments for more details. This targets iTextSharp 5.4.2.0.
//Folder that we're working from
var workingFolder = Environment.GetFolderPath(Environment.SpecialFolder.Desktop);
//Create a regular non-conformant PDF, nothing special below
var RegularPdf = Path.Combine(workingFolder, "File1.pdf");
using (var fs = new FileStream(RegularPdf, FileMode.Create, FileAccess.Write, FileShare.None)) {
using (var doc = new Document()) {
using (var writer = PdfWriter.GetInstance(doc, fs)) {
doc.Open();
doc.Add(new Paragraph("Hello world!"));
doc.Close();
}
}
}
//Create our conformant document from the above file
var ConformantPdf = Path.Combine(workingFolder, "File2.pdf");
using (var fs = new FileStream(ConformantPdf, FileMode.Create, FileAccess.Write, FileShare.None)) {
using (var doc = new Document()) {
//Use PdfSmartCopy to get every page
using (var copy = new PdfSmartCopy(doc, fs)) {
//Set our conformance levels
copy.SetPdfVersion(PdfWriter.PDF_VERSION_1_3);
copy.PDFXConformance = PdfWriter.PDFX1A2001;
//Open our new document for writing
doc.Open();
//Bring in every page from the old PDF
using (var r = new PdfReader(RegularPdf)) {
for (var i = 1; i <= r.NumberOfPages; i++) {
copy.AddPage(copy.GetImportedPage(r, i));
}
}
//Close up
doc.Close();
}
}
}
Just to be 100% clear, this WILL NOT MAKE A CONFORMANT PDF, just a document that says it conforms.