iText 7: PdfDictionary released prematurely - pdf

Question

For certain PDFs I see an NPE while flushing the tag structure. This problem happens with iText 7.0.2-SNAPSHOT. iText 5.5.10 handles these files just fine. The NPE is thrown in PdfDictionary.get(PdfName key, boolean asDirect) because the map is null.
The only time the map in that class can become null is when releaseContent() is called.
Since releaseContent()'s only purpose is - as far as I can tell - to free memory, I've tested what happens when it's changed to an empty method.
The result is that the files appear to be processed normally. No more exceptions. Here is an example file.
Only a small number of objects are accessed after being released. For the example file from above, this would fix the problem as well:
protected void releaseContent() {
List<Integer> objs = Arrays.asList(6888, 6856, 6824, 844, 836);
if (objs.contains(indirectReference.objNr)) {
return;
}
map = null;
}
I'll leave the analysis of why this happens to those more knowledgeable than me. Whether this is the result of a broken PDF or a bug in iText7 I do not know.
Input/Output
What I am doing with iText 7.0.2-SNAPSHOT:
PdfReader reader = new PdfReader(src);
PdfWriter writer = new PdfWriter(dest);
PdfDocument doc = new PdfDocument(reader, writer);
doc.close();`
What the output is:
Exception in thread "main" com.itextpdf.kernel.PdfException: Tag structure flushing failed: it might be corrupted.
at com.itextpdf.kernel.pdf.PdfDocument.tryFlushTagStructure(PdfDocument.java:1746)
at com.itextpdf.kernel.pdf.PdfDocument.close(PdfDocument.java:727)
at perinorm.cleanPdf.MainCleanPDF.run(MainCleanPDF.java:139)
at perinorm.cleanPdf.MainCleanPDF.lambda$2(MainCleanPDF.java:58)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source)
at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
at java.util.stream.ReferencePipeline$2$1.accept(Unknown Source)
at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
at java.util.Iterator.forEachRemaining(Unknown Source)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Unknown Source)
at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(Unknown Source)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(Unknown Source)
at java.util.stream.AbstractPipeline.evaluate(Unknown Source)
at java.util.stream.ReferencePipeline.forEach(Unknown Source)
at perinorm.cleanPdf.Main.main(Main.java:56)
Caused by: java.lang.NullPointerException
at com.itextpdf.kernel.pdf.PdfDictionary.get(PdfDictionary.java:555)
at com.itextpdf.kernel.pdf.PdfDictionary.get(PdfDictionary.java:146)
at com.itextpdf.kernel.pdf.tagging.PdfStructElem.getK(PdfStructElem.java:338)
at com.itextpdf.kernel.pdf.tagging.PdfStructElem.getKids(PdfStructElem.java:322)
at com.itextpdf.kernel.pdf.tagging.PdfStructTreeRoot.flushAllKids(PdfStructTreeRoot.java:247)
at com.itextpdf.kernel.pdf.tagging.PdfStructTreeRoot.flushAllKids(PdfStructTreeRoot.java:249)
at com.itextpdf.kernel.pdf.tagging.PdfStructTreeRoot.flushAllKids(PdfStructTreeRoot.java:249)
at com.itextpdf.kernel.pdf.tagging.PdfStructTreeRoot.flushAllKids(PdfStructTreeRoot.java:249)
at com.itextpdf.kernel.pdf.tagging.PdfStructTreeRoot.flushAllKids(PdfStructTreeRoot.java:249)
at com.itextpdf.kernel.pdf.tagging.PdfStructTreeRoot.flushAllKids(PdfStructTreeRoot.java:249)
at com.itextpdf.kernel.pdf.tagging.PdfStructTreeRoot.flush(PdfStructTreeRoot.java:184)
at com.itextpdf.kernel.pdf.PdfDocument.tryFlushTagStructure(PdfDocument.java:1744)
... 16 more

Answer 1

The immediate cause of this issue is that in the structure tree of the sample document some nodes are used more than once, e.g.
6770 0 obj
<<
/K [ 6873 0 R 6874 0 R 6875 0 R 6876 0 R 6877 0 R 6878 0 R 6879 0 R
6880 0 R 6881 0 R 6882 0 R 6883 0 R 6884 0 R 6885 0 R 6886 0 R
6887 0 R 6888 0 R 6888 0 R ]
/P 5874 0 R
/S /TR
>>
As you see the 6888 0 R occurs twice in array of kids of this structure tree node.
When iText 7 closes the PdfDocument, it walks the structure tree and flushes each element it finds to the target document:
private void flushAllKids(IPdfStructElem elem) {
for (IPdfStructElem kid : elem.getKids()) {
if (kid instanceof PdfStructElem) {
flushAllKids(kid);
((PdfStructElem) kid).flush();
}
}
}
(from PdfStructTreeRoot)
In the document at hand, therefore, it flushes the child dictionary object 6888 0 when it finds the first 6888 0 R reference and then fails when it finds the second.
I'm not really into structure trees and, therefore, am not sure whether it is ok to have repeating entries in the structure tree or not (after all it is called a tree which could indicate that a given node shall be referenced in it as a child only once). But one can harden iText against such repeatedly used nodes by changing the method above to
private void flushAllKids(IPdfStructElem elem) {
for (IPdfStructElem kid : elem.getKids()) {
if (kid instanceof PdfStructElem) {
if (!((PdfStructElem) kid).isFlushed())
{
flushAllKids(kid);
((PdfStructElem) kid).flush();
}
}
}
}
With this change the sample document is stamped without an exception.