I have a W9 PDF document that I am filling with data for just a single record per button click. Now the client would like to create a single document where each record is a page in the document. Below is our code to create a PDF for each employee.
protected void lnkFillFields_Click(object sender, EventArgs e)
{
using (Library.Data.PDFData data = new Library.Data.PDFData())
{
try
{
Document document = new Document();
PdfCopy writer = new PdfCopy(document, Response.OutputStream);
document.Open();
foreach (EmployeeData emp in data.sp_select_employee_data())
{
//Creates a PDF from a byte array
PdfReader reader =
new PdfReader((Byte[])data.sp_select_doc(16).Tables[0].Rows[0]["doc"]);
//Creates a "stamper" object used to populate interactive fields
MemoryStream ms = new MemoryStream();
PdfStamper stamper = new PdfStamper(reader, ms);
try
{
//MUST HAVE HERE BEFORE STREAMING!!!
//This line populates the interactive fields with your data.
// false = Keeps the fields as editable
// true = Turns all of the editable fields to their read-only equivalent
stamper.FormFlattening = false;
//fill in PDF here
stamper.Close();
reader.Close();
MergePDFs(writer, ms);
}
catch (Exception ex)
{
throw ex;
}
}
document.Close();
//Stream the file to the user
Response.ContentType = "application/pdf";
Response.BufferOutput = true;
Response.AppendHeader("Content-Disposition", "attachment; filename=W9"+ "_Complete.pdf");
Response.Flush();
Response.End();
}
catch (Exception ex)
{
throw ex;
}
}
}
Inserting a page wasn't the way to go. Instead, merging the documents was essentially what we wanted. Therefore, we have come up with this method:
UPDATE
Below is the method that we came up with that successfully stitches a new PDF to the previous one.
private static void MergePDFs(PdfCopy writer, MemoryStream ms)
{
PdfReader populated_reader = new PdfReader(ms.ToArray());
//Add this pdf to the combined writer
int n = populated_reader.NumberOfPages;
for (int i = 1; i <= n; i++)
{
PdfImportedPage page = writer.GetImportedPage(populated_reader, i);
writer.AddPage(page);
}
}
What we need to do is create all of this in memory, then spit it out to the user for download.
Check out kuujinbo's tutorial here for combining/stitching PDFs together.
Before you do that, you'll also need to obviously generate the PDFs, too. You might be tempted to try to do it all in one pass which will work but will be harder to debug. Instead, I'd recommend making two passes, the first to create the individual documents and the second to combine them. Your first pass can temporarily writer them to either disk or memory. Your code (and kuujinbo's) actually writes directly to the Response stream which is completely valid, too, but also much harder to debug, especially if you wrap everything in a giant try/catch.
The number of PDFs you're joining and the frequency of generation should determine where you're temporarily storing the first pass two. If you're only doing up to a dozen or two and they're not giant I would persist to a MemoryStream and work with the .ToArray() byte data on that.
If you've got more PDFs than that or they're fairly large or this routine gets called often or you have RAM constraints you might be better persisting them to a unique folder first, stitching them and then deleting that folder.
Related
I am having trouble merging PDFs in-memory. I have 2 memory streams, a master and component stream, the idea is that as each component PDF is built up, the component PDF's bytes are added to the master stream. At the very end of all the components, we have a byte array that's a PDF.
I have the code below, but nothing is copying into my masterStream. I think the issue is with CopyPagesTo, but I'm not familiar enough and the documentation/examples are hard to find.
byte[] updated;
using (MemoryStream masterMemoryStream = new MemoryStream())
{
masterStream.WriteTo(masterMemoryStream);
// Read from master stream (ie. all existing components)
masterMemoryStream.Position = 0;
using (iText.Kernel.Pdf.PdfWriter masterPdfWriter = new iText.Kernel.Pdf.PdfWriter(masterMemoryStream))
using (iText.Kernel.Pdf.PdfDocument masterPdfDocument = new iText.Kernel.Pdf.PdfDocument(masterPdfWriter))
{
using (MemoryStream componentMemoryStream = new MemoryStream())
{
componentStream.WriteTo(componentMemoryStream);
// Read from new component
componentMemoryStream.Position = 0;
using (iText.Kernel.Pdf.PdfReader componentPdfReader = new iText.Kernel.Pdf.PdfReader(componentMemoryStream))
using (iText.Kernel.Pdf.PdfDocument componentPdfDocument = new iText.Kernel.Pdf.PdfDocument(componentPdfReader))
{
// Copy pages from component into master
componentPdfDocument.CopyPagesTo(1, componentPdfDocument.GetNumberOfPages(), masterPdfDocument);
}
}
}
updated = masterMemoryStream.GetBuffer();
}
// Write updates to master stream?
masterStream.SetLength(0);
using (MemoryStream temp = new MemoryStream(updated))
temp.WriteTo(masterStream);
Answer
This is mkl's answer with some of my corrections:
using (MemoryStream temporaryStream = new MemoryStream())
{
masterStream.Position = 0;
componentStream.Position = 0;
using (PdfDocument combinedDocument = new PdfDocument(new PdfReader(masterStream), new PdfWriter(temporaryStream)))
using (PdfDocument componentDocument = new PdfDocument(new PdfReader(componentStream)))
{
componentDocument.CopyPagesTo(1, componentDocument.GetNumberOfPages(), combinedDocument);
}
byte[] temporaryBytes = temporaryStream.ToArray();
masterStream.Position = 0;
masterStream.SetLength(temporaryBytes.Length);
masterStream.Capacity = temporaryBytes.Length;
masterStream.Write(temporaryBytes, 0, temporaryBytes.Length);
}
There are a number of issues in your code. I'll first give you a working version and then go into the issues in your code.
A working version (with an important limitation)
You can combine two PDFs given in MemoryStream instances masterStream and componentStream and get the result in the same MemoryStream instance masterStream as follows:
using (MemoryStream temporaryStream = new MemoryStream())
{
masterStream.Position = 0;
componentStream.Position = 0;
using (PdfDocument combinedDocument = new PdfDocument(new PdfReader(masterStream), new PdfWriter(temporaryStream)))
using (PdfDocument componentDocument = new PdfDocument(new PdfReader(componentStream)))
{
componentDocument.CopyPagesTo(1, componentDocument.GetNumberOfPages(), combinedDocument);
}
byte[] temporaryBytes = temporaryStream.ToArray();
masterStream.Position = 0;
masterStream.Capacity = temporaryBytes.Length;
masterStream.Write(temporaryBytes, 0, temporaryBytes.Length);
masterStream.Position = 0;
}
The limitation is that you have to have instantiated the masterStream with an expandable capacity; the MemoryStream class has a number of constructors only some of which create such an expandable instance while the others create non-resizable instances. For details read here.
Issues in your concept and code
Concatenating PDF files does not result in a valid merged PDF
You describe your concept like this
the idea is that as each component PDF is built up, the component PDF's bytes are added to the master stream
This does not work, though, the PDF format does not allow merging PDFs by simply concatenating them. In particular the (active) objects in a PDF have an identifier number which must be unique in the PDF, concatenating would result in a file with non-unique object identifiers; PDFs contain cross reference structures which map each object identifier to its offset from the file start, concatenating would get all these offsets wrong for the added PDFs; furthermore, a PDF has to have a single root object from which the other objects are referenced directly or indirectly, concatenating would result in multiple root objects.
Writing and immediately overwriting
In your code you have
masterStream.WriteTo(masterMemoryStream);
// Read from master stream (ie. all existing components)
masterMemoryStream.Position = 0;
using (iText.Kernel.Pdf.PdfWriter masterPdfWriter = new iText.Kernel.Pdf.PdfWriter(masterMemoryStream))
Here you write the contents of masterStream to masterMemoryStream, then set the masterMemoryStream position to the start and instantiate a PdfWriter which starts writing there. I.e. your original copy of the masterStream contents get overwritten, surely not what you wanted.
Using MemoryStream.GetBuffer
MemoryStream.GetBuffer does not only return the data written into the MemoryStream by design but the whole buffer; i.e. there may be a lot of trash bytes after the actual PDF in what you retrieve here
updated = masterMemoryStream.GetBuffer();
This may cause PDF processors trying to process your result PDFs to be unable to open the file: PDFs have a pointer to the last cross references at their end, so if you have trash bytes following the actual end of your PDF, PDF processors may not find that pointer.
PS
As worked out in the comments, the code above works fine in case of constantly growing stream lengths (which usually will happen in the use case at hand) but in general one needs to restrict the stream size before writing the new content, e.g. like this:
...
masterStream.Position = 0;
masterStream.SetLength(temporaryBytes.Length); // <<<<
masterStream.Capacity = temporaryBytes.Length;
...
Currently I am using a custom LocationTextExtractionStrategy to extract text from a PDF that returns a TextRenderInfo[]. I would like to be able to determine if a TextRenderInfo object (or PDFString, child of TextRenderInfo) appears in a specific layer. I am not sure if this is possible. To get the layers in a PDF, I am using:
Dictionary<string,PdfLayer> layers;
using (var pdfReader = new PdfReader(src))
{
var newSrc = Path.Combine(["new file location"]);
using (var stream = new FileStream(newSrc, FileMode.Create))
{
PdfStamper stamper = new PdfStamper(pdfReader, stream);
layers = stamper.GetPdfLayers();
stamper.Close();
}
pdfReader.Close();
src = newSrc;
}
To extract the text, I am using:
var textExtractor = new TextExtractionStrategy();
PdfTextExtractor.GetTextFromPage(pdfReader, pdfPageNum,textExtractor);
List<TextRenderInfo> results = textExtractor.Results;
Is there any way that I can check if the individual TextRenderInfo results exist within the layers obtained in the first code snippet. Any help would be much appreciated.
It is possible to get the contents from a single layer, but you'll have to jump through a few hoops to work it out. Specifically, you will have to recreate some of the logic that is provided by the PdfTextExtractor and PdfReaderContentParser.
public static String GetText(PdfReader reader, int pageNumber, int streamNumber) {
var strategy = new LocationTextExtractionStrategy();
var processor = new PdfContentStreamProcessor(strategy);
var resourcesDic = pageDic.GetAsDict(PdfName.RESOURCES);
// assuming you still need to extract the page bytes
byte[] contents = GetContentBytesForPageStream(reader, pageNumber, streamNumber);
processor.ProcessContent(contents, resourcesDic);
return strategy.GetResultantText();
}
public static byte[] GetContentBytesForPageStream(PdfReader reader, int pageNumber, int streamNumber) {
PdfDictionary pageDictionary = reader.GetPageN(pageNum);
PdfObject contentObject = pageDictionary.Get(PdfName.CONTENTS);
if (contentObject == null)
return new byte[0];
byte[] contentBytes = GetContentBytesFromContentObject(contentObject, streamNumber);
return contentBytes;
}
public static byte[] GetContentBytesFromContentObject(PdfObject contentObject, int streamNumber) {
// copy-paste logic from
// ContentByteUtils.GetContentBytesFromContentObject(contentObject);
// but in case PdfObject.ARRAY: only select the streamNumber you require
}
If you're specifically looking to just use PdfTextExtractor or PdfReaderContentParser, and ask the returned TextRenderInfo for the layer it's on, then I'm not sure it will be easily possible. There are a number of problems with that:
TextRenderInfo doesn't store that information, so you'd have to subclass it (which is possible)
you'd have to rewrite the logic that creates the TextRenderInfo objects. This is possible by registering custom IContentOperator objects for all text operators (Tj, TJ, ' and ") with the PdfTextExtractor or PdfReaderContentParser
the hardest part is that you have already lost layer information in ContentByteUtils.GetContentBytesFromContentObject - so you'd need to retain that somehow, which creates its own set of problems.
I am trying to split shared XObject streams (originally flatten form fields with the same content) in the PDF.
What is the correct way to do this using itextsharp? I am trying the code below but the stream is still shared in the resulting document.
Sample pdf with shared XObject streams flatten.pdf
PdfReader pdf = new PdfReader(path);
PdfStamper stamper = new PdfStamper(pdf, new FileStream("processed.pdf", FileMode.OpenOrCreate, FileAccess.ReadWrite));
EliminateSharedStreams(stamper, 1);
stamper.Close();
virtual public void EliminateSharedXObjectStreams(PdfStamper pdfStamper, int pageNum)
{
PdfReader pdfReader = pdfStamper.Reader;
PdfDictionary page = pdfReader.GetPageN(pageNum);
PdfDictionary resources = page.GetAsDict(PdfName.RESOURCES);
PdfDictionary xObjResources = resources.GetAsDict(PdfName.XOBJECT);
List<PRIndirectReference> newRefs = new List<PRIndirectReference>();
List<PdfName> newNames = new List<PdfName>();
List<PRStream> newStreams = new List<PRStream>();
IntHashtable visited = new IntHashtable();
foreach (PdfName key in xObjResources.Keys)
{
PdfStream xObj = xObjResources.GetAsStream(key);
if (xObj is PRStream && xObj.GetAsName(PdfName.SUBTYPE) != null &&
xObj.GetAsName(PdfName.SUBTYPE).CompareTo(PdfName.FORM) == 0)
{
PRIndirectReference refi = (PRIndirectReference)xObjResources.Get(key);
PRStream xFormStream = (PRStream)xObj;
if (visited.ContainsKey(refi.Number))
{
// need to duplicate
newRefs.Add(refi);
PRStream newStream = new PRStream(xFormStream, null);
newStreams.Add(newStream);
newNames.Add(key);
}
else
visited[xFormStream.ObjNum] = 1;
}
}
if (newStreams.Count == 0)
return;
PdfContentByte canvas = pdfStamper.GetOverContent(pageNum);
PdfWriter writer = pdfStamper.Writer;
for (int k = 0; k < newStreams.Count; ++k)
{
canvas.SaveState();
//add copied stream
PdfIndirectReference newRef = writer.AddToBody(newStreams[k]).IndirectReference;
//change the ref
xObjResources.Put(newNames[k], newRef);
canvas.RestoreState();
}
}
First remarks without a sample document
There are numerous reasons why your code may not work as desired. As you did not supply your sample PDF, I cannot tell which are more relevant and which are not.
You only search xobjects shared on the same page; if a xobject is once used on page one and once on page two, your code cannot identify this.
If you want to be able to find such shares, you'll at least have to use the same IntHashtable instance visited across all calls of EliminateSharedXObjectStreams for the same PdfStamper pdfStamper, e.g. by creating it once outside this method and making it a parameter of your method.
You only check for shared xobjects in the immediate page resources. But form xobjects have their own resources which can contain even more form xobject declarations.
If you want to find such shares, you'll have to recurse into the resources of your page's xobjects, those xobjects' xobjects, etc. pp.
(Strictly speaking you also have to recurse into the form xobjects of patterns and Type 3 Font glyph definitions, but these are unlikely positions to flatten form fields into.)
You only check for shared xobjects with different names. But xobjects can also be shared by referencing the same name multiple times from the same content stream.
If you want to find such shares, you have to analyse the content streams in question to find duplicate usages of the form xobject with the same name.
(By the way, doing so you may also check whether declared xobjects are used at all: if a form xobject is declared in some resources, this does not mean it is used in the context of these resources, it may be an unused resource.)
You don't mark xObjResources (if it itself is indirect) or page (otherwise) as used. If your PdfStamper pdfStamper is working in append mode, your changes may be ignored.
Solution with a sample document
After you provided the information that
It's single page document containing shared streams (xobjects with different names) in the immediate page resources. pdfStamper is not in append mode.
it turned out that the problems mentioned above are not relevant in your case. As you meanwhile also have provided an example document, I could reproduce the issue.
Indeed, your code does not split the shared XObjects. The reason is that the PdfStamper is made for manipulating the PDF in the PdfReader in the state it was in when the stamper was constructed, using stamper methods only. Your code, on the other hand, manipulates objects directly retrieved from the PdfReader after the construction of the stamper. Thus, while your new streams are added to the PDF (actually up front), the changes in the pre-existing XObject resource dictionaries don't make it to the result.
If you want to manipulate objects you retrieve from the reader, you instead should do this before creating the stamper.
This actually should suite you as your code structurally is copied from a Pdfreader method anyways, EliminateSharedStreams, which you adapted to your use case.
The only problem is that that method uses a hidden member variable of the PdfReader class. But you can access that variable bei means of reflection.
Thus, the manipulated method (working on a pure PdfReader) could look like this:
virtual public void EliminateSharedXObjectStreams(PdfReader pdfReader, int pageNum)
{
PdfDictionary page = pdfReader.GetPageN(pageNum);
PdfDictionary resources = page.GetAsDict(PdfName.RESOURCES);
PdfDictionary xObjResources = resources.GetAsDict(PdfName.XOBJECT);
List<PRIndirectReference> newRefs = new List<PRIndirectReference>();
List<PRStream> newStreams = new List<PRStream>();
IntHashtable visited = new IntHashtable();
foreach (PdfName key in xObjResources.Keys)
{
PdfStream xObj = xObjResources.GetAsStream(key);
if (xObj is PRStream && xObj.GetAsName(PdfName.SUBTYPE) != null &&
xObj.GetAsName(PdfName.SUBTYPE).CompareTo(PdfName.FORM) == 0)
{
PRIndirectReference refi = (PRIndirectReference)xObjResources.Get(key);
PRStream xFormStream = (PRStream)xObj;
if (visited.ContainsKey(refi.Number))
{
// need to duplicate
newRefs.Add(refi);
PRStream newStream = new PRStream(xFormStream, null);
newStreams.Add(newStream);
}
else
visited[xFormStream.ObjNum] = 1;
}
}
if (newStreams.Count == 0)
return;
FieldInfo xrefObjField = typeof(PdfReader).GetField("xrefObj", BindingFlags.Instance | BindingFlags.NonPublic);
List<PdfObject> xrefObj = (List<PdfObject>)xrefObjField.GetValue(pdfReader);
for (int k = 0; k < newStreams.Count; ++k)
{
xrefObj.Add(newStreams[k]);
PRIndirectReference refi = newRefs[k];
refi.SetNumber(xrefObj.Count - 1, 0);
}
}
and you can use it like this:
using (PdfReader pdfReader = new PdfReader(sourcePath))
using (Stream pdfStream = new FileStream(targetPath, FileMode.Create, FileAccess.Write))
{
EliminateSharedXObjectStreams(pdfReader, 1);
PdfStamper pdfStamper = new PdfStamper(pdfReader, pdfStream);
pdfStamper.Close();
}
in particular calling EliminateSharedXObjectStreams before constructing the PdfStamper.
If you are after a generic solution, you of course will have to extend the method to remove the restrictions observed in the first part of the answer...
Solution without reflection
The OP found out:
Manipulating PdfReader works as expected. Only thing is that instead of using xrefObj private field, the stream can be add using AddPdfObject:
for (int k = 0; k < newStreams.Count; ++k)
{
PRIndirectReference newRef = pdfReader.AddPdfObject(newStreams[k]);
PRIndirectReference refi = newRefs[k];
refi.SetNumber(newRef.Number, 0);
}
Indeed, this improves the solution substantially.
Please help me understand if my solution is correct.
I'm trying to extract text from a PDF file with a LocationTextExtractionStrategy parser. I'm getting exceptions because the ParseContentMethod tries to parse inline images? The code is simple and looks similar to this:
RenderFilter[] filter = { new RegionTextRenderFilter(cropBox) };
ITextExtractionStrategy strategy = new FilteredTextRenderListener(new LocationTextExtractionStrategy(), filter);
PdfTextExtractor.GetTextFromPage(pdfReader, pageNumber, strategy);
I realize the images are in the content stream but I have a PDF file failing to extract text because of inline images. It returns an UnsupportedPdfException of "The filter /DCTDECODE is not supported" and then it finally fails with and InlineImageParseException of "Could not find image data or EI", when all I really care about is the text. The BI/EI exists in my file so I assume this failure is because of the /DCTDECODE exception. But again, I don't care about images, I'm looking for text.
My current solution for this is to add a filterHandler in the InlineImageUtils class that assigns the Filter_DoNothing() filter to the DCTDECODE filterHandler dictionary. This way I don't get exceptions when I have InlineImages with DCTDECODE. Like this:
private static bool InlineImageStreamBytesAreComplete(byte[] samples, PdfDictionary imageDictionary) {
try {
IDictionary<PdfName, FilterHandlers.IFilterHandler> handlers = new Dictionary<PdfName, FilterHandlers.IFilterHandler>(FilterHandlers.GetDefaultFilterHandlers());
handlers[PdfName.DCTDECODE] = new Filter_DoNothing();
PdfReader.DecodeBytes(samples, imageDictionary, handlers);
return true;
} catch (IOException e) {
return false;
}
}
public class Filter_DoNothing : FilterHandlers.IFilterHandler
{
public byte[] Decode(byte[] b, PdfName filterName, PdfObject decodeParams, PdfDictionary streamDictionary)
{
return b;
}
}
My problem with this "fix" is that I had to change the iTextSharp library. I'd rather not do that so I can try to stay compatible with future versions.
Here's the PDF in question:
https://app.box.com/s/7eaewzu4mnby9ogpl2frzjswgqxn9rz5
I am getting a pdf-document (no password) which is generated from a third party software with javascript and a few editable fields in it. If I load this pdf-document with the pdfReader class the NumberOfPagesProperty is always 1 although the pdf-document has 17 pages. Oddly enough the document has 17 pages if I save the stream afterwards. When I now try to open the document the Acrobat Reader shows an extended feature warning and the fields are not fillable anymore (I haven't flattened the document). Do anyone know about such a problem?
Background Info:
My job is to remove the javascript code, fill out some fields and save the document afterwards.
I am using the iTextsharp version 5.5.3.0.
Unfortunately I can't upload a sample file because there are some confidental data in it.
private byte[] GetDocumentData(string documentName)
{
var document = String.Format("{0}{1}\\{2}.pdf", _component.OutputDirectory, _component.OutputFileName.Replace(".xml", ".pdf"), documentName);
if (File.Exists(document))
{
PdfReader.unethicalreading = true;
using (var originalData = new MemoryStream(File.ReadAllBytes(document)))
{
using (var updatedData = new MemoryStream())
{
var pdfTool = new PdfInserter(originalData, updatedData) {FormFlattening = false};
pdfTool.RemoveJavascript();
pdfTool.Save();
return updatedData.ToArray();
}
}
}
return null;
}
//Old version that wasn't working
public PdfInserter(Stream pdfInputStream, Stream pdfOutputStream)
{
_pdfInputStream = pdfInputStream;
_pdfOutputStream = pdfOutputStream;
_pdfReader = new PdfReader(_pdfInputStream);
_pdfStamper = new PdfStamper(_pdfReader, _pdfOutputStream);
}
//Solution
public PdfInserter(Stream pdfInputStream, Stream pdfOutputStream, char pdfVersion = '\0', bool append = true)
{
_pdfInputStream = pdfInputStream;
_pdfOutputStream = pdfOutputStream;
_pdfReader = new PdfReader(_pdfInputStream);
_pdfStamper = new PdfStamper(_pdfReader, _pdfOutputStream, pdfVersion, append);
}
public void RemoveJavascript()
{
for (int i = 0; i <= _pdfReader.XrefSize; i++)
{
PdfDictionary dictionary = _pdfReader.GetPdfObject(i) as PdfDictionary;
if (dictionary != null)
{
dictionary.Remove(PdfName.AA);
dictionary.Remove(PdfName.JS);
dictionary.Remove(PdfName.JAVASCRIPT);
}
}
}
The extended feature warning is a hint that the original PDF had been signed using a usage rights signature to "Reader-enable" it, i.e. to tell the Adobe Reader to activate some additional features when opening it, and the OP's operation on it has invalidated the signature.
Indeed, he operated using
_pdfStamper = new PdfStamper(_pdfReader, _pdfOutputStream);
which creates a PdfStamper which completely re-generates the document. To not invalidate the signature, though, one has to use append mode as in the OP's fixed code (for char pdfVersion = '\0', bool append = true):
_pdfStamper = new PdfStamper(_pdfReader, _pdfOutputStream, pdfVersion, append);
If I load this pdf-document with the pdfReader class the NumberOfPagesProperty is always 1 although the pdf-document has 17 pages. Oddly enough the document has 17 pages
Quite likely it is a PDF with a XFA form, i.e. the PDF is only a carrier of some XFA data from which Adobe Reader builds those 17 pages. The actual PDF in that case usually only contains one page saying something like "if you see this, your viewer does not support XFA."
For a final verdict, though, one has to inspect the PDF.