What is the recommended way to import the standard cryptographic hashing (message digest) libraries (MD5, SHA1, SHA2, SHA256, SHA3, etc.) in Ceylon?
There doesn't seem to be a cryptography module in the SDK.
There is Ceylon Crypto on Github (and as 3 separate modules in Herd), but it says in the README:
Note that IANAC (I am not a cryptologist), so this will surely be flawed in some security relevant way.
Do not use in production and don't rely on it in any expensive way whatsoever!
If you just want to use this in the JVM, I would suggest to use just Java's crypto APIs in java.security (that should be enough for the hash functions) or javax.crypto (for other stuff like ciphers).
Here is an example, calculating the SHA-256 of Hello World:
module.ceylon:
native("jvm")
module example "1.0.0" {
import java.base "8";
}
run.ceylon:
import java.security {
MessageDigest
}
import java.lang {
JString=String,
ByteArray
}
"Formats a single byte as a (two character) hexadecimal string."
String formatByte(Byte byte)
=> Integer.format(byte.unsigned, 16).padLeading(2, '0');
"Formats a Java byte array as a hexadecimal string."
String formatByteArray(ByteArray result)
=> String.sum(result.byteArray.map(formatByte));
"Calculates SHA-256('Hello World') and print it."
shared void run() {
value message = "Hello World";
value bytes = JString(message).getBytes("UTF-8");
value dig = MessageDigest.getInstance("SHA-256");
value result = dig.digest(bytes);
value formatted = formatByteArray(result);
print("Result: ``result.array```");
print("Length: ``result.size``");
print("Result in hex: ``formatted``");
}
This program outputs this:
Result: { -91, -111, -90, -44, 11, -12, 32, 64, 74, 1, 23, 51, -49, -73, -79, -112, -42, 44, 101, -65, 11, -51, -93, 43, 87, -78, 119, -39, -83, -97, 20, 110 }`
Length: 32
Result in hex: A591A6D40BF420404A011733CFB7B190D62C65BF0BCDA32B57B277D9AD9F146E
I didn't find a Ceylon wrapper for this which would make it a bit nicer, though.
Working with the 2016-W4 pdf, which has 2 large streams (page 1 & 2), along with a bunch of other objects and smaller streams. I'm trying to deflate the stream(s), to work with the source data, but am struggling. I'm only able to get corrupt inputs and invalid checksums errors.
I've written a test script to help debug, and have pulled out smaller streams from the file to test with.
Here are 2 streams from the original pdf, along with their length objects:
stream 1:
149 0 obj
<< /Length 150 0 R /Filter /FlateDecode /Type /XObject /Subtype /Form /FormType
1 /BBox [0 0 8 8] /Resources 151 0 R >>
stream
x+TT(T0�B ,JUWÈS0Ð37±402V(NFJS�þ¶
«
endstream
endobj
150 0 obj
42
endobj
stream 2
142 0 obj
<< /Length 143 0 R /Filter /FlateDecode /Type /XObject /Subtype /Form /FormType
1 /BBox [0 0 0 0] /Resources 144 0 R >>
stream
x+T�ç�ã
endstream
endobj
143 0 obj
11
endobj
I copied just the stream contents into new files within Vim (excluding the carriage returns after stream and before endstream).
I've tried both:
compress/flate (rfc-1951) – (removing the first 2 bytes (CMF, FLG))
compress/zlib (rfc-1950)
I've converted the streams to []byte for the below:
package main
import (
"bytes"
"compress/flate"
"compress/gzip"
"compress/zlib"
"fmt"
"io"
"os"
)
var (
flateReaderFn = func(r io.Reader) (io.ReadCloser, error) { return flate.NewReader(r), nil }
zlibReaderFn = func(r io.Reader) (io.ReadCloser, error) { return zlib.NewReader(r) }
)
func deflate(b []byte, skip, length int, newReader func(io.Reader) (io.ReadCloser, error)) {
// rfc-1950
// --------
// First 2 bytes
// [120, 1] - CMF, FLG
//
// CMF: 120
// 0111 1000
// ↑ ↑
// | CM(8) = deflate compression method
// CINFO(7) = 32k LZ77 window size
//
// FLG: 1
// 0001 ← FCHECK
// (CMF*256 + FLG) % 31 == 0
// 120 * 256 + 1 = 30721
// 30721 % 31 == 0
stream := bytes.NewReader(b[skip:length])
r, err := newReader(stream)
if err != nil {
fmt.Println("\nfailed to create reader,", err)
return
}
n, err := io.Copy(os.Stdout, r)
if err != nil {
if n > 0 {
fmt.Print("\n")
}
fmt.Println("\nfailed to write contents from reader,", err)
return
}
fmt.Printf("%d bytes written\n", n)
r.Close()
}
func main() {
//readerFn, skip := flateReaderFn, 2 // compress/flate RFC-1951, ignore first 2 bytes
readerFn, skip := zlibReaderFn, 0 // compress/zlib RFC-1950, ignore nothing
// ⤹ This is where the error occurs: `flate: corrupt input before offset 19`.
stream1 := []byte{120, 1, 43, 84, 8, 84, 40, 84, 48, 0, 66, 11, 32, 44, 74, 85, 8, 87, 195, 136, 83, 48, 195, 144, 51, 55, 194, 177, 52, 48, 50, 86, 40, 78, 70, 194, 150, 74, 83, 8, 4, 0, 195, 190, 194, 182, 10, 194, 171, 10}
stream2 := []byte{120, 1, 43, 84, 8, 4, 0, 1, 195, 167, 0, 195, 163, 10}
fmt.Println("----------------------------------------\nStream 1:")
deflate(stream1, skip, 42, readerFn) // flate: corrupt input before offset 19
fmt.Println("----------------------------------------\nStream 2:")
deflate(stream2, skip, 11, readerFn) // invalid checksum
}
I'm sure I'm doing something wrong somewhere, I just can't quite see it.
(The pdf does open in a viewer)
Binary data should never be copied out of / saved from text editors. There might be cases when this succeeds, and it just adds oil to the flame.
Your data that you eventually "mined out" from the PDF is most likely not identical to the actual data that is in the PDF. You should take the data from a hex editor (e.g. try hecate for something new), or write a simple app that saves it (which strictly handles the file as binary).
Hint #1:
The binary data displayed spread across multiple lines. Binary data does not contain carriage returns, that's a textual control. If it does, that means the editor did interpret it as text, and so some codes / characters where "consumed" to start a new line. Multiple sequences may be interpreted as the same newline (e.g. \n, \r\n). By excluding them, you're already at data loss, by including them, you might already have a different sequence. And if the data was interpreted and displayed as text, more problems may arise as there are more control characters, and some characters may not appear when displayed.
Hint #2:
When flateReaderFn is used, decoding the 2nd example succeeds (completes without an error). This means "you were barking up the right tree", but the success depends on what the actual data is and to what extent was it "distorted" by the text editor.
Okay, confession time...
I was so caught up in trying to understand deflate that I completely overlooked the fact that Vim wasn't saving the stream contents correctly into new files. So I spent quite a bit of time reading the RFC's, and digging through the internals of the Go compress/... packages, assuming the problem was with my code.
Shortly after I posted my question I tried reading the PDF as a whole, finding the stream/endstream locations, and pushing that through deflate. As soon as I saw the content scroll through the screen I realized my dumb mistake.
+1 #icza, that was exactly my issue.
It was good in then end, as I have a much better understanding of the whole process than if it would have just worked the first go around.
Extracting objects from PDF can be tricky depending on the filters used. The filter can also have additional options which need to be handled correctly.
For someone interested in extracting an object without taking care of the low-level details of the process.
To get a single object from a PDF and decode it can be done as follows:
package main
import (
"fmt"
"os"
"strconv"
"github.com/unidoc/unipdf/v3/core"
"github.com/unidoc/unipdf/v3/model"
)
func main() {
objNum := 149 // Get object 149
err := inspectPdfObject("input.pdf", objNum)
if err != nil {
fmt.Printf("Error: %v\n", err)
os.Exit(1)
}
}
func inspectPdfObject(inputPath string, objNum int) error {
f, err := os.Open(inputPath)
if err != nil {
return err
}
defer f.Close()
pdfReader, err := model.NewPdfReader(f)
if err != nil {
return err
}
isEncrypted, err := pdfReader.IsEncrypted()
if err != nil {
return err
}
if isEncrypted {
// If encrypted, try decrypting with an empty one.
// Can also specify a user/owner password here by modifying the line below.
auth, err := pdfReader.Decrypt([]byte(""))
if err != nil {
fmt.Printf("Decryption error: %v\n", err)
return err
}
if !auth {
fmt.Println(" This file is encrypted with opening password. Modify the code to specify the password.")
return nil
}
}
obj, err := pdfReader.GetIndirectObjectByNumber(objNum)
if err != nil {
return err
}
fmt.Printf("Object %d: %s\n", objNum, obj.String())
if stream, is := obj.(*core.PdfObjectStream); is {
decoded, err := core.DecodeStream(stream)
if err != nil {
return err
}
fmt.Printf("Decoded:\n%s", decoded)
} else if indObj, is := obj.(*core.PdfIndirectObject); is {
fmt.Printf("%T\n", indObj.PdfObject)
fmt.Printf("%s\n", indObj.PdfObject.String())
}
return nil
}
A full example: pdf_get_object.go
Disclosure: I am the original developer of UniPDF.
I am getting below for the first time giving print command.
"An error exists on this page. Acrobat may not display the page correctly. please
contact the person who created the pdf document to correct the problem".
Print out is comming very fine. and Second time print out command not giving any error.
Please help me why this error is comming for the first time print.
This is part of my code to create PDF
PdfContentByte cb = writer.DirectContent;
cb.BeginText();
Font NormalFont = FontFactory.GetFont("Arial", 12, Font.NORMAL, Color.BLACK);
// Add an image to a fixed position
iTextSharp.text.Image img = iTextSharp.text.Image.GetInstance(Server.MapPath("~/images/images/banner.tiff"));
img.SetAbsolutePosition(35, 760);
img.ScalePercent(50);
cb.AddImage(img);
// Draw a line by setting the line width and position
cb.SetLineWidth(2);
cb.MoveTo(20, 740);
cb.LineTo(570, 740);
cb.Stroke();
//Header Details
cb.BeginText();
writeText(cb, drHead["EmpName"].ToString(), 25, 745, f_cb, 14);
writeText(cb, "Employee ID:", 450, 745, f_cn, 12);
writeText(cb, drHead["EmployeeID"].ToString(), 515, 745, f_cb, 12);
cb.EndText();
cb.BeginText();
writeText(cb, "XXXX:", 25, 725, f_cb, 8);
cb.EndText();
cb.SetLineWidth(2);
cb.MoveTo(20, 675);
cb.LineTo(570, 675);
cb.Stroke();
cb.EndText();
// Acknowledgement section
cb.BeginText();
writeText(cb, "XXXXXXXXXXXXXXXX", 20, 140, f_cb, 12);
cb.EndText();
cb.EndText();
Please help me to know what is the issue.
You have nested text blocks. That's illegal PDF syntax. I think recent versions of iTextSharp warn you about this, so I guess you're using an old version.
This is wrong:
cb.BeginText();
...
cb.BeginText();
...
cb.EndText();
...
cb.EndText();
This is right:
cb.BeginText();
...
cb.EndText();
...
cb.BeginText();
...
cb.EndText();
Moreover: ISO-32000-1 tells you that some operations are forbidden inside a text block.
This is wrong:
cb.BeginText();
...
cb.AddImage(img);
...
cb.EndText();
This is right:
cb.BeginText();
...
cb.EndText();
...
cb.AddImage(img);
Finally, some operators are mandatory when creating a text block. For instance: you always need setFontAndSize() (I don't know what you're doing in writeText(), but I assume you're setting the font correctly).
In any case: you have chosen to use iTextSharp at the lowest level, writing PDF syntax almost manually. This assumes that you know ISO-32000-1 inside-out. If you don't, you should use some of the high-level objects, such as ColumnText to position content at absolute positions.
I'm trying to pass a two-dimensional array, which size can be dynamic, as a method argument.
Within the method I'd like to use the array with the general array syntax.
int item = array[row][column];
To pass the array is not possible, so I thought about to use a pointer pointer.
- (void)doSomethingWithArray:(int **)array columns:(int)nColumns rows:(int)nRows
{
int item = array[n][m];
}
But I get the problem when I try to pass the array as the parameter
int array[numberOfRows][numberOfColumns];
[someObject doSomethingWithArray:array columns:numberOfColumns rows:numberOfRows];
I found a lot of tips & tricks, but somehow nothing really works in the way I would like to use it.
Thanks for help,
Eny
Is objective-c based on C99?
If it is, you can use the "new" syntax that allows you to pass dimension information directly.
#include <stdio.h>
void foo(int rows, int cols, int arr[rows][cols]) {
printf("%d, %d\n", arr[0][0], arr[1][4]);
}
int main(void) {
int arr[2][12] = {{1, 2, 3, 4, 5}, {11, 12, 13, 14, 15}};
foo(2, 12, arr);
}
You can see the code running on ideone.
- (void)doSomethingWithArray:(void *)array columns:(int)nColumns rows:(int)nRows {}
...
[someObject doSomethingWithArray:&array columns:numberOfColumns rows:numberOfRows];