Collision Attacks, Message Digests and a Possible solution - cryptography

I've been doing some preliminary research in the area of message digests. Specifically collision attacks of cryptographic hash functions such as MD5 and SHA-1, such as the Postscript example and X.509 certificate duplicate.
From what I can tell in the case of the postscript attack, specific data was generated and embedded within the header of the postscript file (which is ignored during rendering) which brought about the internal state of the md5 to a state such that the modified wording of the document would lead to a final MD value equivalent to the original postscript file.
The X.509 took a similar approach where by data was injected within the comment/whitespace sections of the certificate.
Ok so here is my question, and I can't seem to find anyone asking this question:
Why isn't the length of ONLY the data being consumed added as a final block to the MD calculation?
In the case of X.509 - Why is the whitespace and comments being taken into account as part of the MD?
Wouldn't a simple processes such as one of the following be enough to resolve the proposed collision attacks:
MD(M + |M|) = xyz
MD(M + |M| + |M| * magicseed_0 +...+ |M| * magicseed_n) = xyz
where :
M : is the message
|M| : size of the message
MD : is the message digest function (eg: md5, sha, whirlpool etc)
xyz : is the pairing of the acutal message digest value for the message M and |M|. <M,|M|>
magicseed_{i}: Is a set of random values generated with seed based on the internal-state prior to the size being added.
This technqiue should work, as to date all such collision attacks rely on adding more data to the original message.
In short, the level of difficulty involved in generating a collision message such that:
It not only generates the same MD
But is also comprehensible/parsible/compliant
and is also the same size as the original message,
is immensely difficult if not near impossible. Has this approach ever been discussed? Any links to papers etc would be nice.
Further Question: What is the lower bound for collisions of messages of common length for a hash function H chosen randomly from U, where U is the set of universal hash functions ?
Is it 1/N (where N is 2^(|M|)) or is it greater? If it is greater, that implies there is more than 1 message of length N that will map to the same MD value for a given H.
If that is the case, how practical is it to find these other messages? bruteforce would be of O(2^N), is there a method of time complexity less than bruteforce?

Can't speak for the rest of the questions, but the first one is fairly simple - adding length data to the input of the md5, at any stage of the hashing process (1st block, Nth block, final block) just changes the output hash. You couldn't retrieve that length from the output hash string afterwards. It's also not inconceivable that a collision couldn't be produced from another string with the exact same length in the first place, so saying "the original string was 17 bytes" is meaningless, because the colliding string could also be 17 bytes.
e.g.
md5("abce(17bytes)fghi") = md5("abdefghi<long sequence of text to produce collision>")
is still possible.

In the case of X.509 certificates specifically, the "comments" are not comments in the programming language sense: they are simply additional attributes with an OID that indicates they are to be interpreted as comments. The signature on a certificate is defined to be over the DER representation of the entire tbsCertificate ('to be signed' certificate) structure which includes all the additional attributes.
Hash function design is pretty deep theory, though, and might be better served on the Theoretical CS Stack Exchange.
As #Marc points out, though, as long as more bits can be modified than the output of the hash function contains, then by the pigeonhole principle a collision must exist for some pair of inputs. Because cryptographic hash functions are in general designed to behave pseudo-randomly over their inputs, collisions will tend toward being uniformly distributed over possible inputs.
EDIT: Incorporating the message length into the final block of the hash function would be equivalent to appending the length of everything that has gone before to the input message, so there's no real need to modify the hash function to do this itself; rather, specify it as part of the usage in a given context. I can see where this would make some types of collision attacks harder to pull off, since if you change the message length there's a changed field "downstream" of the area modified by the attack. However, this wouldn't necessarily impede the X.509 intermediate CA forgery attack since the length of the tbsCertificate is not modified.

Related

Is 128-bit md5 do full mapping for 128-bit binary set?

I'm writing a utility to convert md5 (or sha1) digest to a distinguishable image, something like ssh-keygen -lv. Usually, similar messages can have digests very different, but hackers can modify the message bit by bit to try to get a similar but still different md5 digest to mock the original one. When matching is done by machine, the trick will certainly fail. But when matching is done by human eye, user could be fooled.
To avoid of such trick, the convert program can generate the image from a modified digest as follow:
image = generateImage( md5(md5 + Random_Secret) )
The Random_Secret will reshape the digest, the similarity introduced by hacker will be removed after the transformation.
Now comes the question, since the final md5() take input of another md5 variable, which is only 128-bit length, (here ignore the Random_Secret which is a constant in all) is it safe to generate enough different values for feeding generateImage()?
Question also for other digest algorithms: sha1, etc.
A hash function should have the following properties, among others:
In cryptography, the avalanche effect is the desirable property of cryptographic algorithms, typically block ciphers and cryptographic hash functions, wherein if an input is changed slightly (for example, flipping a single bit), the output changes significantly (e.g., half the output bits flip).
see https://en.wikipedia.org/wiki/Avalanche_effect
Thus, hackers should not be able to slightly modify the message to obtain a similar digest unless the hash function is broken. Also it should not be possible to create a message that results in a specific hash value (so it should resist preimage attacks, see https://en.wikipedia.org/wiki/Preimage_attack).
md5 is cryptographically broken
So would the use of a simple md5 hash already be sufficient?
No. Although md5 is a widely used hash function, it has the problem that it is cryptographically broken and therefore insecure.
One basic requirement of any cryptographic hash function is that it should be computationally infeasible to find two distinct messages that hash to the same value. MD5 fails this requirement catastrophically; such collisions can be found in seconds on an ordinary home computer.
see https://en.wikipedia.org/wiki/MD5
Therefore, it is generally recommended to stop using md5 in a cryptographic context.
Since such a collision would result in the same hash in the first inner hash operation in your approach, the repeated hash with an additional constant random secret would also be the same. This means that this attack can successfully exchange the message or file with a different one.
The other hash algorithm you mentioned, SHA-1, is also cryptographically broken.
In this context, there is a worth reading article from Arstechnica from 2008 about the exploitation of md5 collisions to create bogus CA intermediate certificates.
Example of a hash collision
Finally to illustrate a hash collision, here are two .jpg files with the same md5 hash. The collision was created using the following open source project published on Github: https://github.com/cr-marcstevens/hashclash.
The following small Python program writes the files yes.jpg and no.jpg into the current directory to be able to compare the files visually, and then calculates the md5 hash for them - which results in exactly the same value for both files.
import binascii
import hashlib
yes = b'ffd8ffe000104a46494600010101012c012c0000ffe100a04578696600004d4d002a000000080005011a0005000000010000004a011b0005000000010000005201280003000000010002000001320002000000140000005a87690004000000010000006e000000000000012c000000010000012c00000001323032323a31313a31392032323a35373a3133000003a00100030000000100010000a00200030000000100800000a0030003000000010080000000000000ffe10c3b687474703a2f2f6e732e61646f62652e636f6d2f7861702f312e302f003c3f787061636b657420626567696e3d22efbbbf222069643d2257354d304d7043656869487a7265537a4e54637a6b633964223f3e203c783a786d706d65746120786d6c6e733a783d2261646f62653a6e733a6d6574612f2220783a786d70746b3d22584d5020436f726520352e352e30223e203c7264663a52444620786d6c6e733a7264663d22687474703a2f2f7777772e77332e6f72672f313939392f30322f32322d7264662d73796e7461782d6e7323223e203c7264663a4465736372697074696f6e207264663a61626f75743d222220786d6c6e733a64633d22687474703a2f2f7075726c2e6f72672f64632f656c656d656e74732f312e312f2220786d6c6e733a70686f746f73686f703d22687474703a2f2f6e732e61646f62652e636f6d2f70686f746f73686f702f312e302f2220786d6c6e733a786d703d22687474703a2f2f6e732e61646f62652e636f6d2f7861702f312e302f2220786d6c6e733a786d704d4d3d22687474703a2f2f6e732e61646f62652e636f6d2f7861702f312e302f6d6d2f2220786d6c6e733a73744576743d22687474703a2f2f6e732e61646f62652e636f6d2f7861702f312e302f73547970652f5265736f757263654576656e7423222070686f746f73686f703a436f6c6f724d6f64653d2233222070686f746f73686f703a49434350726f66696c653d22735247422049454336313936362d322e312220786d703a4d65746164617461446174653d22323032322d31312d31395432323a35373a31332b30313a30302220786d703a4d6f64696679446174653d22323032322d31312d31395432323a35373a31332b30313a3030223e203c64633a7469746c653e203c7264663a416c743e203c7264663a6c6920786d6c3a6c616e673d22782d64656661756c74223e7965733c2f7264663a6c693e203c2f7264663a416c743e203c2f64633a7469746c653e203c786d704d4d3a486973746f72793e203c7264663a5365713e203c7264663a6c6920786d704d4d3a616374696f6e3d2270726f64756365642220786d704d4d3a736f6674776172654167656e743d22416666696e6974792044657369676e657220312e31302e352220786d704d4d3a7768656e3d22323032322d31312d31395432303a34343a33372b30313a3030222f3e203c7264663a6c692073744576743a616374696f6e3d2270726f6475636564222073744576743a736f6674776172654167656e743d22416666696e6974792050686f746f20312e31302e35222073744576743a7768656e3d22323032322d31312d31395432323a35373a31332b30313a3030222f3e203c2f7264663a5365713e203c2f786d704d4d3a486973746f72793e203c2f7264663a4465736372697074696f6e3e203c2f7264663a5244463e203c2f783a786d706d6574613e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020203c3f787061636b657420656e643d2277223f3effed005050686f746f73686f7020332e30003842494d04040000000000171c015a00031b25471c0200000200041c02050003796573003842494d0425000000000010acd1c50e9eee8829b1943af6ed4ce40dffe202644943435f50524f46494c45000101000002546c636d73043000006d6e74725247422058595a2007e6000b0013000f00240031616373704150504c0000000000000000000000000000000000000000000000000000f6d6000100000000d32d6c636d7300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000b64657363000001080000003e63707274000001480000004c77747074000001940000001463686164000001a80000002c7258595a000001d4000000146258595a000001e8000000146758595a000001fc000000147254524300000210000000206754524300000210000000206254524300000210000000206368726d00000230000000246d6c756300000000000000010000000c656e5553000000220000001c0073005200470042002000490045004300360031003900360036002d0032002e003100006d6c756300000000000000010000000c656e5553000000300000001c004e006f00200063006f0070007900720069006700680074002c002000750073006500200066007200650065006c007958595a20000000000000f6d6000100000000d32d736633320000000000010c42000005defffff325000007930000fd90fffffba1fffffda2000003dc0000c06e58595a200000000000006fa0000038f50000039058595a20000000000000249f00000f840000b6c358595a2000000000000062970000b787000018d9706172610000000000030000000266660000f2a700000d59000013d000000a5b6368726d00000000000300000000a3d70000547b00004ccd0000999a0000266600000f5cffdb004300100b0c0e0c0a100e0d0e1211101318281a181616183123251d283a333d3c3933383740485c4e404457453738506d51575f626768673e4d71797064785c656763ffdb0043011112121815182f1a1a2f634238426363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363ffc00011080080008003012200021101031101ffc4001b00010002030101000000000000000000000005060102030407ffc40035100002020102040306030803000000000000010203040511122131410651611314223271e152a1b116344262818291926372f0ffc4001a010100020301000000000000000000000000020501030406ffc40023110101010002010401050000000000000000010203110412213151131432414252ffda000c03010002110311003f00fa00000000000000000020752d7ecc3cf9515d519c21b716ef9b64be1e5579b8d1beaf965d9f54fc884de756c8d58e6c6f5732fbc7700136d000000000000000000000d6728c20e5269452ddb7d84e71ae0e73928c62b76df62a7aceb52cc6e9a1b8d0babef3fb1af939262775a39f9f3c39eefca3b3eff79cdbae5d2726d7d3b161f09efee97f971adbfc1572efa2e27b9e9d5c24b69cbe397d59cbc12eb7ea56f852eb9bd4f7800ee5c800000000000000006b39c6b839cda8c62b76df636203c4da82854b0eb97c53e73dbb2f223bd4c67bad5cdc938b175517abead667d8eb8371c78be4bf17ab23002b35ababdd506f7adebd5a7b349a617ea745767cae5bbf5db99793e769b8b4e2da6b9a6892a75ecfaa1c3ed2334bbce3bb37f0f2e713aaebf17c8c714b351730543f6933ff00e2ff005fb9df17c4d72b12caae1283eae0b668df3c8c576cf3b8ade96806232528a945ee9add1937bb0000000003593e18b93e896e6c70cd970e15f25dab97e82b1abd4b555cad7f36f93f67354c37e4a2b9ff00922e5294e4e5393949f56deed980556b5ad7cd79cdf26b77bd5ec001140000036842564e3082de527b25ea6a7b747ba18fa9d13b229c78b6fa6fcb72599dd92a5992ea4abae3d6eac7aeb7d6314bf23a18325abd249d4e800064000038e5c78b12e8f9c24bf23b1ceffddecffabfd0c5f8475fb6be7a002a5e68000000003dda3e27be6a35d6e4928fc6fd52ec78493f0f46c7ab54e09b4b7e27e4b627c73bd46de292f2665fb5c8c805a3d100000000073bd6f8f625de2ff43a180c59dce9f3a05b323c378b75b2b236595f13df65b34709785a1fc39525f5815f7c7da92f85cd3f85681607e16b3b6547fd3ee693f0be42f92faa5f54d18fc3c9f48df179a7f5410261f86f3974753feefb185e1bce6f9ba97f77d88fe2dfd23fa7e5ff00351318b949462b76dec92ee5f30712ac3c78d75414797c4fbb7ea47699a057896abaf9ab6c8fca92e4993475f0715c7bd58f87e3de3ef5b9ee000e8778000000000000693b6baf6f6938c37e9c4f6dcdcf06b3a6c354d3ecc796ca7f3572fc32ec07b6528c22e53928c5756dec8d1e4d1149caead26b75bc97329d56466eb8b1b45ba33add127ef537dd45f2ff00de7b1d3c495e1d1af69f5e45129e2c28e17557befb7c5b6db35e805b619145925185d5ca4fa2524d8b3229a5a56dd5c1be8a524b72bda22d127a941e0e9f914df14dc676716cb96cfac9f99199355387a9e5cb5ec0bef8dd6375e4464f64bb6dcfe9f402f09a6b74f74cc90fe198515e97c38b992caa78df0b947670fe5dbf3fea4c000000000000000000000109a5e064e3ebfa964db5f0d376dece5c49effd3a9c75ac4d45ebd899f838aaf54d6d34e718addefe6fd4b0802230f335ab32ab8656995d34b7f14d5d17b72f2dcf1463afe9d75f54295a8d1649b84ecb79c7d1eecb20021bc37a65fa7635d2c9e18db7d9c6e10e90f426400000000003ffd9c4b3388e3f1f25df001a62f92687ecd5a5e418c4696dc90500000000189c71a5bddcd3a76b848dfd728e06107cfc7c58590457b174269b47a819677378d674e75717b38cb4c1d899c835393e77ef8ac23a6906f23d7a1ca0c44587a1a53265b96bedce9336efec51ff4915b409013aaceb0063ca664f64f24b8aa1845f0ad97a608304e6bdd2dbf22402569ab6d864ed484f1328cba836c425d8bf1a3c5cd8fd2a4e73fe5cb6359576f0e08597d74cc34405281ff880335050a3aecd75509ea44f847da805e2e5f2b51f05af0d1832d8905e7a7dd7a2227c412e241bf800fdb09c97bc11f6c412842e7102a793a894527be3a7ab503784dc33138fd2b2fc13ff48af9abd222c17c218f4d79f2b4a736622d67efad4e6e2794eeca6cae5983eb5eef36704a91578ec5f82c5dd9efb87e5e51b289c05eb6aefa9ca7b4e252839d299dcb24711aa196fd5b6a9af9147e137e1df9e778ccb8e8eede6a4814e59c957402cd6684eabf560277fe6195d635331a55005dd639de303c51d5ca468e6de7d5718f48a00dc4cce9476feda338a6b65bf702c368df0cdc71676b44c829a1054a3c7585dc3927bd9dca3660b6b4f078a85663f7ad5e0157412dd0d17aa43f5983441a66ed6900672f35bf14515698b98d173b7bdfc639c5cbcab9eec9ab30bb3f055536f'
no = b'ffd8ffe000104a46494600010101012c012c0000ffe100a04578696600004d4d002a000000080005011a0005000000010000004a011b0005000000010000005201280003000000010002000001320002000000140000005a87690004000000010000006e000000000000012c000000010000012c00000001323032323a31313a31392032323a35363a3331000003a00100030000000100010000a00200030000000100800000a0030003000000010080000000000000ffe10c3b687474703a2f2f6e732e61646f62652e636f6d2f7861702f312e302f003c3f787061636b657420626567696e3d22efbbbf222069643d2257354d304d7043656869487a7265537a4e54637a6b633964223f3e203c783a786d706d65746120786d6c6e733a783d2261646f62653a6e733a6d6574612f2220783a786d70746b3d22584d5020436f726520352e352e30223e203c7264663a52444620786d6c6e733a7264663d22687474703a2f2f7777772e77332e6f72672f313939392f30322f32322d7264662d73796e7461782d6e7323223e203c7264663a4465736372697074696f6e207264663a61626f75743d222220786d6c6e733a64633d22687474703a2f2f7075726c2e6f72672f64632f656c656d656e74732f312e312f2220786d6c6e733a70686f746f73686f703d22687474703a2f2f6e732e61646f62652e636f6d2f70686f746f73686f702f312e302f2220786d6c6e733a786d703d22687474703a2f2f6e732e61646f62652e636f6d2f7861702f312e302f2220786d6c6e733a786d704d4d3d22687474703a2f2f6e732e61646f62652e636f6d2f7861702f312e302f6d6d2f2220786d6c6e733a73744576743d22687474703a2f2f6e732e61646f62652e636f6d2f7861702f312e302f73547970652f5265736f757263654576656e7423222070686f746f73686f703a436f6c6f724d6f64653d2233222070686f746f73686f703a49434350726f66696c653d22735247422049454336313936362d322e312220786d703a4d65746164617461446174653d22323032322d31312d31395432323a35363a33312b30313a30302220786d703a4d6f64696679446174653d22323032322d31312d31395432323a35363a33312b30313a3030223e203c64633a7469746c653e203c7264663a416c743e203c7264663a6c6920786d6c3a6c616e673d22782d64656661756c74223e7965733c2f7264663a6c693e203c2f7264663a416c743e203c2f64633a7469746c653e203c786d704d4d3a486973746f72793e203c7264663a5365713e203c7264663a6c6920786d704d4d3a616374696f6e3d2270726f64756365642220786d704d4d3a736f6674776172654167656e743d22416666696e6974792044657369676e657220312e31302e352220786d704d4d3a7768656e3d22323032322d31312d31395432303a34343a32332b30313a3030222f3e203c7264663a6c692073744576743a616374696f6e3d2270726f6475636564222073744576743a736f6674776172654167656e743d22416666696e6974792050686f746f20312e31302e35222073744576743a7768656e3d22323032322d31312d31395432323a35363a33312b30313a3030222f3e203c2f7264663a5365713e203c2f786d704d4d3a486973746f72793e203c2f7264663a4465736372697074696f6e3e203c2f7264663a5244463e203c2f783a786d706d6574613e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020203c3f787061636b657420656e643d2277223f3effed005050686f746f73686f7020332e30003842494d04040000000000171c015a00031b25471c0200000200041c02050003796573003842494d0425000000000010acd1c50e9eee8829b1943af6ed4ce40dffe202644943435f50524f46494c45000101000002546c636d73043000006d6e74725247422058595a2007e6000b0013000f00240031616373704150504c0000000000000000000000000000000000000000000000000000f6d6000100000000d32d6c636d7300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000b64657363000001080000003e63707274000001480000004c77747074000001940000001463686164000001a80000002c7258595a000001d4000000146258595a000001e8000000146758595a000001fc000000147254524300000210000000206754524300000210000000206254524300000210000000206368726d00000230000000246d6c756300000000000000010000000c656e5553000000220000001c0073005200470042002000490045004300360031003900360036002d0032002e003100006d6c756300000000000000010000000c656e5553000000300000001c004e006f00200063006f0070007900720069006700680074002c002000750073006500200066007200650065006c007958595a20000000000000f6d6000100000000d32d736633320000000000010c42000005defffff325000007930000fd90fffffba1fffffda2000003dc0000c06e58595a200000000000006fa0000038f50000039058595a20000000000000249f00000f840000b6c358595a2000000000000062970000b787000018d9706172610000000000030000000266660000f2a700000d59000013d000000a5b6368726d00000000000300000000a3d70000547b00004ccd0000999a0000266600000f5cffdb004300100b0c0e0c0a100e0d0e1211101318281a181616183123251d283a333d3c3933383740485c4e404457453738506d51575f626768673e4d71797064785c656763ffdb0043011112121815182f1a1a2f634238426363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363ffc00011080080008003012200021101031101ffc4001b00010001050100000000000000000000000006020304050701ffc40038100002020102030406060b000000000000000102030405110612213141517113142281a1b13242526191d1151623344472a2c1d2e1f0ffc4001a010100020301000000000000000000000000040501020306ffc40029110100020103020309010000000000000000010203041131125105214113141522324281a1b1e1ffda000c03010002110311003f00e80000001e01e835193c47838f64a1bd964a2f67cb1eff00798ef8af17bb1eef87e6739cb48f548ae973da378a4b7e08ecb8b29fa98b37e7248b32e2d9fd5c38fbe7fe8d673e3eee91a1d44fdbfc4a01139f15e4bfa18f547cdb6790e2bc95f4f1ea7e5ba31ef18fbb6f876a3b7ee12d06a34bd7a8cfb15528ba6e7d89bdd4bc99b73ad6d168de113263be3b74de3690006cd0000000002ddf2e5c7b24bb545bf8170b791fbbdbfc8fe42598e5cdfb7b400533d9000000002a84e55ce3383da517ba7e0ce8789935e563c2daa6a69a5bb5e273a36dc356db0d5ab85726a13df9d7735b1234f93a6db775778869fdae3ebdfcea9b000b179d000000000b3972e5c4ba5e15c9fc0bc63e72e6c0c84bbeb97c8c4f0cd79873b0014ef64000000001b0d0b2e387a9d739c778cbd8f2dfbcd799da363acad528ae4d28f3733ebdbb75d8de9bf546ce39e2b38add5c6d29e9e9e1e96cf240000000014ce3cd0945f7ad8a80104cbd0f3b1652fd8cac82ec943aefee35cd34da6b668e9846b8a74d5cab3aa8ecd74b36f8320e5d3c563aaabcd2f88cdef14c91cfaa3000222dc00cfab46d46e82943167cafb39b65f3331599e21a5f2529f54ecc02ba6d9d1742daded383dd3360f87f535fc3ff005afccc9c4e19cbb2c8fac38d55efd7aeecde315f7f2871beab045677b4259459e9b1ebb3edc54bf145c2984542118456ca2b64545abcac8000000000000516d70baa95762e68496cd78a2b0041357d2add36e7d1ca893f627fd9fde6b8e917535e4552aad829c24b66990dd6b46b34f9bb2bde78edf47df1fb995f9b074fcd5e17fa3d7464da9939feff00ac6d1acaebd5b1e56a5c9cdb75f17d9f127c73427da3667aee9b55adfb6972cbcd1d34b6e6ae3e2b8a77ae4fc338004c53000000000000000000000516d70b6b9576454a125b34fbcac01ceb3b1fd5736ea3ec4da5e5dc493841bf53c85dde917c8b7aaf0fe4e5ea53be99d6abb366dc9f55dc6ef4ec2af4fc48d15f5dbaca5e2fc4878b15ab9267d16faad5d3269eb589ded3b6eca001315000000000000000000000a1590763ad4e2e6bab8efd57b8f7d243d27273c79f6df977ebf8108d52dc9c4e2ccccec65cdeadc92b23e30718a7f33674645797c618f914cb9abb30f993fc40915b7554adedb2105e3292455194671528c94a2fb1a7b914d13069e20bb2f51d479ae5e95d75c1c9a515dbdde68bb835fe86e2a5a763ce5eab935f3c6b6f7e47d7fc58128000000000000000000000001a4c7d3aefd64d4726ea53c5bea8c22db4d4ba4535b76f7330747d072b4de22958a2e58718c9573725d13ea96dbee4a40119af0755d132f21e9d4432f12e973a839a8b83f7991a569b9b66ab3d57545085dcbc95d507bf22ff00be66f800000000000001ffd9037527a862a8e061ce54b290f08b2542358ec9d82b5909f87d37c425a383f7991a569b9b66ab3d57545085dcbc95d507bf22ff00be66f800000000000001ffd9037527a862a8e061ce54b290f08b254235000000006a9bf440bd18cc346b848dfd728e06107cfc7c58590457b174269b47a819677378d674e75717b38cb4c1d899c835393e77ef8ac2366906f23d7a1ca0c44587a1a53265b96bedce9336efec51ff4915b409013aaceb0063ca664f64f24b8aa1845f0ad97a608304e6bdd2dbf22402569ab6d864ed484f3328cba836c425d8bf1a3c5cd8fd2a4e73fe5cb6359576f0e08597d74cc34405281ff880335050a3aecd75509ea44f847da805e2e5f2b51f05af0d1832d8905e7abdd7a2227c412e241bf800fdb09c97bc11f6c412842e7102a793a894527be3a7ab503784dc33138fd2b2fc13ff48af9abd222c17c218f4d79f2b4a736622f67efad4e6e2794eeca6cae5983eb5eef36704a91578ec5f82c5dd9efb87e5e51b289c05eb6aefa9ca7b4e252839d299dcb24711aa196fd5b6a9af9147e137e1df9e788ccb8e8eede6a4814e59c957402cd6684eabf560277fe6195d635331a55005dd639de303c51d5ca468e6de7d5718f48a00dc4cce9476feda338a6b65bf712c368df0cdc71676b44c829a1054a3c7585dc3927bd9dca3660b6b4f078a85663f7ad5e0157412dd0d17aa43f5983441a66ed6900672f35bf14515698b98d173b9bdfc639c5cbcab9eec9ab30bb3f055536f'
def write_file(filename, data):
with open(filename, 'wb') as f:
f.write(data)
if __name__ == '__main__':
yes_data = binascii.unhexlify(yes)
write_file("yes.jpg", yes_data)
no_data = binascii.unhexlify(no)
write_file("no.jpg", no_data)
md5_yes = hashlib.md5(yes_data).hexdigest()
md5_no = hashlib.md5(no_data).hexdigest()
print("yes.jpg, md5 =", md5_yes)
print("no.jpg, md5 =", md5_no)

What kind of password hash/encryption is this?

Trying to find out what kind of hash/encryption this is in a SQL db. We want to create an app to do password resets but I cannot for the life of me find how they are generating this value.
Here are multiple examples
{enc:1}gdyb21LQTcIANtvYMT7QVQ==
{enc:1}ZEImYhrd/Ot/LcewJCFdMw==
{enc:1}+IOSBzegAx7nSytO1J3GEw==
{enc:1}6ULa1IFMxavY7SE66K3UDg==
{enc:1}UEFeGim2M8d0Iee7ejFRUw==
{enc:1}RjqL5rXOBpqJAKPjYkwLtw==
{enc:1}V/PEslecyYxFki03H4ctAQ==
{enc:1}VDEm9QmD+L7tsqcPz/S9XQ==
{enc:1}YkQuEPpL6dRfDLwKMEVMMg==
{enc:1}86rG96EP7T3tx9e8C7513g==
{enc:1}xvSwvsmkgwtXwVISvg7mJQ==
{enc:1}KjCF0RD4tcUGEP4Bpafw5A==
{enc:1}z9pGrdm4QAoomrFCJCXQIQ==
{enc:1}yhLv0HeW90FENKJjV9Nb+g==
{enc:1}EtYMl5FOW+zdpVvWsIj3Rw==
Let's take it apart:
{enc:1}gdyb21LQTcIANtvYMT7QVQ==
The {enc:1} part is likely to be versioning of some form. This is reasonably common so that you can upgrade whatever hashing/encryption algorithm you use over time. Each value identifies the algorithm used to produce the hash - if the implementation decides to use a different approach, it would know to validate a password using the first version, but could then replace the {enc:1} with {enc:2} or whatever, along with the new form of the hash.
The rest is base64 - and it's 24 characters ending in ==, which means the original value is 16 bytes.
So it's a 16 byte (128 bit) hash of some kind, with a versioning prefix. That's pretty much all we can tell... it could be any 128 bit hash, or possibly 128 bits of a longer hash (although you'd have to wonder why they threw away data at that point).
You could perform further tests if you can create your own users with passwords. For example, do two users with the same password end up with the same hash? If not, if you change from password X to password Y and then back to password X for a single user, does that get to the same hash? (There's no obvious salt in the value, although the username could be used as the salt.)

RSACryptoPad gives different results each time

I'm trying to test my RSA implementation's correctness with the RSACryptoPAD example in here: http://www.codeproject.com/Articles/10877/Public-Key-RSA-Encryption-in-C-NET
But it always creates different encryption results. Isn't RSA just a mod and power operation? But the program can decrypt all different encrypted texts correctly. My results are same with the http://nmichaels.org/rsa.py site. I think RSACryptoPAD is doing some other things?
The code uses RSACryptoServiceProvider. The line
byte[] encryptedBytes = rsaCryptoServiceProvider.Encrypt( tempBytes, true );
Tells it to encrypt using OAEP padding, which introduces randomness into the padding. The reason for doing this is so that encrypting the same plaintext will always yield different results (as you are seeing). This is a good thing, as it stops an information leak where an attacker sees you sent the same message multiple times.
There's a great historical example of why this is important. "AF is short of water"

Are RSA signatures unique?

I want to know if RSA signatures are unique for a data.
Suppose I have a "hello" string. The method of computing the RSA signature is firstly to get the sha1 digest(these are , I know, unqiue for data), then add a header with OID and padding scheme mentioned and do some mathematical jiggle to give the signature.
Now assuming padding is same, will the signature generating by openSSL or Bouncy Castle be same?
If yes, my only fear is, won't it be easy to get back the "text"/data??
I actaully tried to do an RSA signature of some data and the signatures from OpenSSL and BC was different. I repeated it but got same signature again and again for each of them. I realized that the two signatures of the methods were different because of the difference in padding. However I am still not sure why the signatures of each of the libs are same all the time I repeat them. Can somebody please give an easy explanation?
The "usual" padding scheme, described in PKCS#1 as the "old-style, v1.5" padding, is deterministic. It works like this:
The data to sign is hashed (e.g. with SHA-1).
A fixed header is added; that header is actually an ASN.1 structure which identifies the hash function which was just used to process the data.
Padding bytes are added (on the left): 0x00, then 0x01, then some 0xFF bytes, then 0x00. The number of 0xFF bytes is adjusted so that the resulting total length is exactly the byte length of the modulus (i.e. 128 bytes for a 1024-bit RSA key).
The padded value is converted to an integer (which is less than the modulus), which goes through the modular exponentiation which is at the core of RSA. The result is converted back to a sequence of bytes, and that's the signature.
All these operations are deterministic, there is no random, hence it is normal and expected that signing the same data with the same key and the same hash function will yield the same signature ever and ever.
However there is a slight underspecification in the ASN.1-based fixed header. This is a structure which identifies the hash function, along with "parameters" for that hash function. Usual hash functions take no parameters, hence the parameters shall be represented with either a special "NULL" value (which takes a few bytes), or be omitted altogether: both representations are acceptable (although the former is supposedly preferred). So, the raw effect is that there are two versions of the "fixed header", for a given hash function. OpenSSL and Bouncycastle do not use the same header. However, signature verifiers are supposed to accept both.
PKCS#1 also describes a newer padding scheme, called PSS, which is more complex but with a stronger security proof. PSS includes a bunch of random bytes, so you will get a distinct signature every time.
Signatures are not a privacy mechanism; it's not considered a problem if you can get the plaintext back out. If your message must be kept secret, then encrypt as well as sign.
Nevertheless, remember that RSA signatures are created using a signer's private key. Given such a signature, you can use the signer's public key to "undo" the RSA transform (raise the message's signature to e, mod n) and get out the SHA1 or other hash value that was provided as its input. You still can't undo the hash function to get the input plaintext corresponding to a signature that has become detached from its message.
RSA for encryption is a different matter. Padding methods for encryption here do include random data in order to defeat traffic analysis.
This is why you add a salt/initialisation vector on top of your key. That way it shouldn't be possible to tell which records came from the same plaintext.

What is the difference between a multi-collision and a first or second pre-image attack on a hash function?

What is the difference between a multi-collision in a hash function and a first or second preimage.
First preimage attacks: given a hash h, find a message m such that
hash(m) = h.
Second preimage attacks: given a fixed message m1, find a different message m2 such that
hash(m2) = hash(m1).
Multi-collision attacks: generate a series of messages m1, m2, ... mN, such that
hash(m1) = hash(m2) = ... = hash(mN).
Wikipedia tells us that a preimage attack differs from a collision attack in that there is a fixed hash or message that is being attacked.
I am confused by papers with which make statements like :
The techniques are
not only efficient to search for
collisions, but also applicable to
explore the second- preimage of MD4.
About the second-preimage attack, they
showed that a random message was a
weak message with probability 2^–122
and it only needed a one-time MD4
computation to find the
second-preimage corresponding to the
weak message.
The Second-Preimage Attack on MD4
If I understand what the authors seem to be saying is that they have developed a multi-collision attack which encompasses a large enough set of messages that given a random message there is a significant though extremely small chance it will overlap with one of their multi-collisions.
I seen similar arguments in many papers. My question when does an attack stop being a multi-collision attack and become a second preimage attack..
If a multi-collision collides with 2^300 other messages does that count as a second preimage, since the multi-collision could be used to calculate the "pre-image" of one of the messages it collides with? Where is the dividing line, 2^60, 2^100, 2^1000?
What if you can generate a preimage of all hash digests that begin with 23? Certainly it doesn't meet the strict definition of a preimage, but it is also very certainly a serious flaw in the cryptographic hash function.
If someone has a large multi-collision, then they could always recover the image of the any message which hash collided with the multi-collision. For instance,
hash(m1) = hash(m2) = hash(m3) = h
Someone requests the preimage of h, and they respond with m2. When does this stop being silly and becomes a real attack?
Rules of thumb? Know of any good resources on evaluating hash function attacks?
Related Links:
HASH COLLISION Q&A
Cryptographic Hashes
The eHash Main Page
It is about an attack scenario. The difference lies in the choice of input. In multi-collision there is free choice of both inputs. 2nd preimage is about finding any second input which has the same output as any specified input.
When a function doesn't have multi-collision resistance, it may be possible to find collision for some kind of messages - not all of them. So this doesn't imply 2nd preimage weakness.
You did a lot of research before posting the question. I cannot answer much aside the resources-question. Which is: I use Applied Cryptography be Menezes/Oorschot for almost everything I ever wanted to know on topics of cryptography, including hashes.
Maybe you'll find a copy at your universities library. Good luck.