Verification of the digital signature of a PE file - cryptography

When checking a file through signtool, I get one hash, and when decoding the certificate, another. What am I missing?
Command:
signtool verify /a /ph /pa /v .\EmptyExe.exe
Hash of file (sha256): 9FCC67FA3FAA88BCDED22E9FCF6AE1D6D62A95A79A1C777743052DF16F63DADC
Decode octet string:
0001f...f003031300d06096086480165030402010500042066cea53b15089957fc4ca86e419e2058f562e17a802e23e7d5154d2e71412e1a
Parsed string: https://lapo.it/asn1js/#MDEwDQYJYIZIAWUDBAIBBQAEIGbOpTsVCJlX_EyobkGeIFj1YuF6gC4j59UVTS5xQS4a
According to my observation, when signing different files, only 3 fields change: hash, messageDigest, encoded hash.
What is stored in the messageDigest field?
Cut digital signature:
https://lapo.it/asn1js/#MIIGRgYJKoZIhvcNAQcCoIIGNzCCBjMCAQExDzANBglghkgBZQMEAgEFADBcBgorBgEEAYI3AgEEoE4wTDAXBgorBgEEAYI3AgEPMAkDAQCgBKICgAAwMTANBglghkgBZQMEAgEFAAQgn8xn-j-qiLze0i6fz2rh1tYqlaeaHHd3QwUt8W9j2tygggPFMIIDwTCCAqmgAwIBAgIQGhQKhwNj5Z5IvU4kiPIzqjANBgkqhkiG9w0BAQsFADBVMVMwUQYDVQQDHkoARwBvAGwAbwB2AGwAZQB2ACAAVABpAG0AbwBmAGUAeQAgACgAVABpAG0AXwBkAGUAdgApACAAMQA1AC4AMAA1AC4AMgAwADAAMDAeFw0yMTAzMDMwMjEzNTJaFw0zOTEyMzEyMzU5NTlaMFUxUzBRBgNVBAMeSgBHAG8AbABvAHYAbABlAHYAIABUAGkAbQBvAGYAZQB5ACAAKABUAGkAbQBfAGQAZQB2ACkAIAAxADUALgAwADUALgAyADAAMAAwMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsdIyw8JcPlDHM1fQGiBKmpWRYRhrOe31xwvYTYaQ02Uc-g0pIGzCu3Q-o6MS0i-2efIKs5shX0HFkLjMy1zgZc2F-PTx8f8HySRxroi5QVngQWLxu638sB9uYdVqBwWyNd7scZx-Z9Fd-kS0rFRIPlyuLCg8UOGtR5KbZ4V7dSNm8myHFTtVqD79n42oJEe2vkmUXQ266B2rHUdHDXJPTiXKwoZg4wAjeTkJUlgJwHeZUvpOkQfoo27C9dh8-4BRR4dHJOtwA1RDyuaVYl1tiQmBAAOcqjKf1bl9u3JLvxldIM8jura2k9oWLA3cxzx7Gr6DlIlGhD7EkyLww3n6VQIDAQABo4GMMIGJMIGGBgNVHQEEfzB9gBBOBdKqObwJh3JpHeW1T741oVcwVTFTMFEGA1UEAx5KAEcAbwBsAG8AdgBsAGUAdgAgAFQAaQBtAG8AZgBlAHkAIAAoAFQAaQBtAF8AZABlAHYAKQAgADEANQAuADAANQAuADIAMAAwADCCEBoUCocDY-WeSL1OJIjyM6owDQYJKoZIhvcNAQELBQADggEBAJeNUpwyUVqKSYGXPj6ibGfs4xxYaHf4obEJ3pgnWFblVgPahzQTutVJ5Ny-TSp0Ger8fTtu9soal35Zz9dpUE9aTYp-YWtEpaaqx5IC-OnH9Cao7ZJ_zM8fwiP9PtHNMuYCBiO24PmHF6oyB0gwcNYh0oa0YaVKJcmtHAVSH6WSzbdea3j9sdlBPVA6FeNchHCCiatesoM75IAUCvKYuBQ9JLenPvCXoKhXBDsiVb5tMKdZD8Vbvoj7b1JzKuv6NkICV99rLWW5MwfRMB-HG-BoML9E2mNJ-kqaVLFbJOZHCaNNIxejR70fY-ijexPNwvr_rI4VW01uYkdmSMlzRLExggH0MIIB8AIBATBpMFUxUzBRBgNVBAMeSgBHAG8AbABvAHYAbABlAHYAIABUAGkAbQBvAGYAZQB5ACAAKABUAGkAbQBfAGQAZQB2ACkAIAAxADUALgAwADUALgAyADAAMAAwAhAaFAqHA2Plnki9TiSI8jOqMA0GCWCGSAFlAwQCAQUAoF4wEAYKKwYBBAGCNwIBDDECMAAwGQYJKoZIhvcNAQkDMQwGCisGAQQBgjcCAQQwLwYJKoZIhvcNAQkEMSIEIBiU-ad7f9l47QV96ioO0N-tTrlArZspsH_A9t1TtmEGMA0GCSqGSIb3DQEBAQUABIIBAC8ocpjX6r77Kr4vEdo4F4PsNFZqxEaugEWpOqwjoYERAZ1QutqyrOebeU9iafvjXAC3chFsgMdpyH4NqW0_Wi1tShhpfhl-fNYatBCXaVbBFYUYuXCdpuXcYtN2nMaO3YR8aQVEKYJKB4UpLTlcho5LmxQDkYy42CZ-L795REpSW_Ts3_6Vcq7ZqBZ7nsRfhOxMOCFqb8gAKBdI8yv2XqZtJeQ258ChkkaUHyKnCWAKXlEVNJC4vhVYJWqBcQhcAdAA1lwHsnl0RklZ1httRb7a5inIH10HrJGypv2E9Zu8knCuQPbRzQvwnwu8k43zOrJnp2DlXspfAB0nxhR6s9w

messageDigest contains a hash of PKCS#7 message content which in turns contains the hash of signed data (PE, in a given case).

Related

what format is this "passcode" file? (found along a keystore)

A Globalprotect vpn client seems to use a keystore in pfx format + a pan_client_cert_passcode.dat file for which I want to find out what the format is (=how to decode it).
I happen to know the passphrase (8 characters), and the passcode file seems to be binary, 16 bytes.
What format could it be?
EDIT:
To help find out: hex dump is 919e **** 77** **73 e18c fd17 d71b 1ddb and known plaintext is VD...ert (each . is a letter, each * is a hex digit)
A Google search for pan_client_cert_passcode dat gave me a result. There is information about this file on paloaltonetworks.com.
It seems you must have received a p12 file from paloalto. And then you create a pfx + dat file from it like this:
globalprotect import-certificate --location cert.p12
See the link. The dat file is definitely a proprietary file format used by the paloalto GlobalProtect program. It's probably encoded because they don't want to store plain text passwords on disk.

computing the exchange hash for ecdsa-sha2-nistp256

I am writing code for an SSH server and can not get past the Elliptic Curve Diffie-Hellman Key Exchange Reply part of the connection. The client also closes the connection and says "Host Key does not match the signature supplied".
I am using putty as the client and a PIC micro-controller is running the server code.
From RFC 5656 [SSH ECC Algorithm Integration] :
"The hash H is formed by applying the algorithm HASH on a
concatenation of the following:
string V_C, client's identification string (CR and LF excluded)
string V_S, server's identification string (CR and LF excluded)
string I_C, payload of the client's SSH_MSG_KEXINIT
string I_S, payload of the server's SSH_MSG_KEXINIT
string K_S, server's public host key
string Q_C, client's ephemeral public key octet string
string Q_S, server's ephemeral public key octet string
mpint K, shared secret
"
the host key algorithm and key exchange algorithm is ecdsa-sha2-nistp256 and ecdh-sha2-nistp256 respectively.
referring to RFC 4251 for data type representations, as well as the source code in openSHH (openBSD) this is what I have concatenated.
4 bytes for then length of V_C followed by V_C
4 bytes for then length of V_S followed by V_S
4 bytes for length of I_C followed by I_C (payload is from Message Code to the start of Random Padding)
4 bytes for length of I_S followed by I_S (payload is from Message Code to the start of Random Padding)
4 bytes for the length of K_S followed by K_S (for K_S I used the same group of bytes that is used to calculate the fingerprint)
4 bytes for the length of Q_C followed by Q_C (i used the uncompressed string which has length of 65 - 04||X-coordinate||Y-coordinate)
4 bytes for the length of Q_S followed by Q_S
4 bytes for the length of K followed by K (length is 32 or 33 depending is the leading bit is set or not. If it is set then K is preceded by a 00 byte)
Once concatenated I hash it with SHA256 because I'm using NISTP256. SHA256 outputs 32 bytes which is the size of the curve, so I take the whole SHA256 output and perform the signature algorithm on it.
I can never get the correct signature from my message concatenation.
I know my signature algorithm is correct because given the message hash output I can get the correct signature.
I know my shared secret is correct because I get the same output as online shared secret calculators.
I know the SHA256 is correct because I get the same result using online calculators.
This leads me to assume the error is in the concatenation of the exchange hash.
Any help is greatly appreciated, thanks.
ECDSA signature generation is non-deterministic, i.e. part of the input is the hash and part of the input consists of random bytes. So whatever you do, you will always get a different signature. This is all right because signature verification will still work.
The only way to get a repeated signature is to mess with the random number generator (during testing, you don't want to sign two values using the same random number: you'd expose the private key!).

Convert X.509 certificate from hex form to .cer format

How to convert X.509 certificate from a hex-dump form to .CER format? Additionally, should be blank space separators removed from hex dump first? Thanks.
You could use ASN.1 editor. It has a data converter that will convert HEX to PEM format of data. the source is also available so if you need to use it in code you can look how it is done.
Or you could use certutil.exe using command
certutil -decodehex c:\cert.hex c:\cert.cer
Normally .cer can either be binary encoded - DER - or it can be DER that has an "ASCII armor", called PEM.
If you want to create DER you probably only have to decode the hexadecimals. In that case you obviously need to disregard any whitespace within the hexadecimals (as well as any other spurious data that may be present).
If you want to have PEM it is required in addition to base 64 encode the result and add a header and footer text. Or you can use an existing library that does this of course.

SSL certificate signature verification

I'm trying to verify certificate on an embedded board manually because it doesn't support Openssl or other libraries. However it does have libraries for MD5 and SHA1 hashes and RSA encryption.
From what I understand to verify the certificate, first compute the SHA1 hash (or MD5) of the certificate; then decode the signature using CA's public key to obtain a hashed value. These two hash values should be the same.
SHA1 hash produces a 20 byte value and MD5 produces 16 byte value. However, RSA decoding of the signature does not. If the CA's key is 1024 bits, the decode signature will be 80bytes; if the CA's key is 512 bits, the decoded signature will be 40 bytes. So I can't really compare the 20 bytes SHA1 value against the 40 or 80 bytes Decoded Signature.
Am I doing something wrong? Or am I missing a step?
The missing bit is the padding algorithm.
You use RSA with a specific padding algorithm, and the same algorithm also tells you how to take the result (when you decrypt the signature) and extract from it the actual data (the hash).

RSA pubkey file type detection

I got a RSA pubkey.dat (almost obvious what it is) that has the following structure on contents:
ASN1 Integer of around 1024 bits (Modulus)
ASN1 Integer (Exponent)
Blob of 256 bytes (Signature)
No tags like "----begin---" or so. pure hex values in it.
There's any way to identify its format like if it's DER/PEM/etc , so i can open it with python crypto libraries or crypto++ on c++?
(Or if it matches a public standard structure name for me to check)
Seems like its not PEM as M2crypt can't load it.
Thanks in advance.
PEM-encoding has mandatory format:
-----BEGIN typeName-----
base64 of DER value
-----END typeName-----
where, for public keys, typeName="PUBLIC KEY" (AFAIR) so that's very easy to check with a regular expression such as the following:
/-----BEGIN [^-]+-----([A-Za-z0-9+\/=\s]+)-----END [^-]+-----/
If it's not PEM, it's usually plain DER.
The DER representation of an ASN.1 SEQUENCE always begins with 0x30 so usually when I have to decode a DER-or-PEM stream which I know for sure it's an ASN.1 SEQUENCE (most complex values are SEQUENCEs, anyways) I check the first byte: if its's 0x30, I decode as DER, else I decode as PEM.
You can check your ASN.1 data quickly using my very own opensource ASN.1 parser (it's all client-side Javascript, so I won't see your data).