I got a RSA pubkey.dat (almost obvious what it is) that has the following structure on contents:
ASN1 Integer of around 1024 bits (Modulus)
ASN1 Integer (Exponent)
Blob of 256 bytes (Signature)
No tags like "----begin---" or so. pure hex values in it.
There's any way to identify its format like if it's DER/PEM/etc , so i can open it with python crypto libraries or crypto++ on c++?
(Or if it matches a public standard structure name for me to check)
Seems like its not PEM as M2crypt can't load it.
Thanks in advance.
PEM-encoding has mandatory format:
-----BEGIN typeName-----
base64 of DER value
-----END typeName-----
where, for public keys, typeName="PUBLIC KEY" (AFAIR) so that's very easy to check with a regular expression such as the following:
/-----BEGIN [^-]+-----([A-Za-z0-9+\/=\s]+)-----END [^-]+-----/
If it's not PEM, it's usually plain DER.
The DER representation of an ASN.1 SEQUENCE always begins with 0x30 so usually when I have to decode a DER-or-PEM stream which I know for sure it's an ASN.1 SEQUENCE (most complex values are SEQUENCEs, anyways) I check the first byte: if its's 0x30, I decode as DER, else I decode as PEM.
You can check your ASN.1 data quickly using my very own opensource ASN.1 parser (it's all client-side Javascript, so I won't see your data).
Related
I need to generate weak certificate for CTF challenge using RSA and small modulus so it's factorable. It should be about 64 bits.
I've tried to generate that using OpenSSL as I would the normal one, but it forbids creating certificate with modulus lower than 512 bits for security reasons. So I've changed the source files of OpenSSL so it doesn't checks the bit length and recompiled that again. I was then able to create private key using smaller modulus, but trying to create certificate using that key (or new small one) evoked new error which I don't fully understand. I even wanted to evade the OpenSSL problem at all using Python, but that just showed that it uses OpenSSL too and had exactly same problems.
Generating small private key:
$ openssl genrsa -out acn.pem 64
-----BEGIN PRIVATE KEY-----
MFQCAQAwDQYJKoZIhvcNAQEBBQAEQDA+AgEAAgkAlz0xJ3uUx5UCAwEAAQIIR1Zs
1Wo4EQECBQDHPJNVAgUAwlPjQQIFAKqNunkCBClt4QECBHlHx1Q=
-----END PRIVATE KEY-----
Generating certificate:
$ openssl req -key acn.pem -new -out domain.csr
...
140561480598848:error:04075070:rsa routines:RSA_sign:digest too big for rsa key:../crypto/rsa/rsa_sign.c:100:
140561480598848:error:0D0DC006:asn1 encoding routines:ASN1_item_sign_ctx:EVP lib:../crypto/asn1/a_sign.c:224:
I found that this thread could be helpful as I could even choose my own numbers, but the method didn't worked for me:
How to Generate rsa keys using specific input numbers in openssl?
Here's sample certificate from PicoCTF. I would like to create similar one.
-----BEGIN CERTIFICATE-----
MIIB6zCB1AICMDkwDQYJKoZIhvcNAQECBQAwEjEQMA4GA1UEAxMHUGljb0NURjAe
Fw0xOTA3MDgwNzIxMThaFw0xOTA2MjYxNzM0MzhaMGcxEDAOBgNVBAsTB1BpY29D
VEYxEDAOBgNVBAoTB1BpY29DVEYxEDAOBgNVBAcTB1BpY29DVEYxEDAOBgNVBAgT
B1BpY29DVEYxCzAJBgNVBAYTAlVTMRAwDgYDVQQDEwdQaWNvQ1RGMCIwDQYJKoZI
hvcNAQEBBQADEQAwDgIHEaTUUhKxfwIDAQABMA0GCSqGSIb3DQEBAgUAA4IBAQAH
al1hMsGeBb3rd/Oq+7uDguueopOvDC864hrpdGubgtjv/hrIsph7FtxM2B4rkkyA
eIV708y31HIplCLruxFdspqvfGvLsCynkYfsY70i6I/dOA6l4Qq/NdmkPDx7edqO
T/zK4jhnRafebqJucXFH8Ak+G6ASNRWhKfFZJTWj5CoyTMIutLU9lDiTXng3rDU1
BhXg04ei1jvAf0UrtpeOA6jUyeCLaKDFRbrOm35xI79r28yO8ng1UAzTRclvkORt
b8LMxw7e+vdIntBGqf7T25PLn/MycGPPvNXyIsTzvvY/MXXJHnAqpI5DlqwzbRHz
q16/S1WLvzg4PsElmv1f
-----END CERTIFICATE-----
I need to generate weak certificate for CTF challenge using RSA and small modulus so it's factorable. It should be about 64 bits.
It's impossible to do that as a self-signed certificate, because proper RSA signing can't work with keys that small.
RSA-SSA-PKCS1_v1.5 is the shortest structured RSA signature padding, and it's structured as (per https://datatracker.ietf.org/doc/html/rfc8017#section-9.2):
DigestInfo ::= SEQUENCE {
digestAlgorithm AlgorithmIdentifier,
digest OCTET STRING
}
The shortest possible encoding for that structure is 9 bytes... and that's with a 0-byte hash algorithm:
30 07 // SEQUENCE (7 content bytes)
30 03 // digestAlgorithm: SEQUENCE (3 bytes)
06 01 00 // OBJECT IDENTIFIER 0.0 ({itu-t(0) recommendation(0)})
// omit implicit NULL
04 00 // digest (empty)
So, with our empty hash we need to keep going:
If emLen < tLen + 11, output "intended encoded message length too short" and stop.
emLen is the length of the modulus (in bytes) and tLen is the length of our encoded structure (9 bytes+).
That makes 20 bytes (160 bits) the shortest possible RSA key to do anything that might stand a chance of being regarded as an RSA signature... which produced a pointless signature (since everything collides under a 0-bit hash).
If you are comfortable stomping on a 1-byte OID for your CTF, your RSA key modulus would need to be 20 bytes + the length of the intended hash (in bytes). Since there's no 1-byte OID that identifies an existing hash algorithm (and no defined hash algorithm for use with certificates that's that small), no existing easy-to-use tool can do this for you.
You could invent a new form of RSA signature padding, of course, using something like a 60-bit hash processed directly with your 64-bit key. That'll require even further work on your part.
You're basically reduced to using a DER writer (or writing your own) to hand-craft a certificate.
Here's sample certificate from PicoCTF. I would like to create similar one.
That certificate is not self-signed. It has a 2048-bit certificate signature, from RSA-SSA-PKCS1_v1.5 with MD2. So while it describes a short RSA key (53 bit modulus) it was signed with something "proper". If that's what you're after, the general flow would be something like
Create a normal self-signed cert
Create your small RSA key
Build a new cert, signed by the first cert, containing the small RSA key.
It's hard to encode a CSR for the small key (because it can't self-sign the request), but maybe there's a way to get openssl req -new to do something other than self-sign, which would allow skipping the intermediate CSR. (Or use library tools like .NET's CertificateRequest, or OpenSSL's APIs instead of their CLI tool, or whatever.)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
Can some one help me understand how an RSA key literally is stored in these formats? I would like to know the difference between the PKCS formats vs Encodings(DER, PEM). From what I understand PEM is more human readable. Is PEM/DER for keys/certs similar to UTF-8/16 for characters? What is the significance of DER/PEM? Sorry too many questions but fed up googling and getting vague answers. Thanks.
(Expanding more than I feel is appropriate for an edit.)
PKCS1, available in several versions as rfcs 2313 2437 3447 and 8017, is primarily about using the RSA algorithm for cryptography including encrypting decrypting signing and verifying. But since crypto is often used between systems or at least programs it is convenient to have a defined, interoperable format for keys, and PKCS1 defines fairly minimal formats for RSA public and private keys in appendix A.1. As Luke implied this uses ASN.1 conventionally encoded as DER, which is a standard for interoperably encoding data of almost any kind.
PKCS8 available as rfc5208 on the other hand is a standard for handling private keys for all algorithms, not just RSA. It also uses ASN.1 DER, and starts by simply combining an AlgorithmIdentifier, an ASN.1 structure (first) defined by X.509 which not very surprisingly identifies an algorithm, with an OCTET STRING which contains a representation of the key in a fashion depending on the algorithm. For algorithm RSA, identified by an AlgorithmIdentifier containing an OID which means rsaEncryption, the OCTET STRING contains the PKCS1 private key encoding.
PKCS8 also allows arbitrary 'attributes' to be added, but this is rarely used.
(E.g. Unable to convert .jks to .pkcs12: excess private key)
PKCS8 also provides an option to encrypt the private key, using password-based encryption (in practice though not explicitly required). This is common, especially when PKCS8 is used as the privatekey portion of PKCS12/PFX, though not universal.
Since most systems today need to support multiple algorithms, and wish to be able to adapt to new algorithms as they are developed, PKCS8 is preferred for privatekeys, and a similar any-algorithm scheme defined by X.509 for publickeys. Although PKCS12/PFX is often preferred to both.
Neither of these has anything to do with certificates or other PKI objects like CSRs, CRLs, OCSP, SCTs, etc. Those are defined by other standards, including some other members of the PKCS series -- although they may use the keys defined by these standards.
PEM format as Luke said is a way of formatting, or (super)encoding, (almost any) binary/DER data in a way that is more convenient. It derives from a 1990s attempt at secure email named Privacy-Enhanced Mail hence PEM. In those days email systems often could transmit, or at least reliably transmit, only printable text with a limited character set, and often only limited line length, so PEM encoded binary data as base64 with line length 64. The PEM scheme itself was not very successful and has been superseded by others like PGP and S/MIME, but the format it defined is still used. Nowadays email systems often can transmit binary data, but as Luke said copy-and-paste often can only handle displayed characters so PEM is still useful, and in addition easier for humans to recognize.
To be more exact, PEM encodes some data, such as but not limited to a PKCS1 or PKCS8 key or a certificate, CSR, etc, as:
a line consisting of 5 hyphens, the word BEGIN, one or a few (space-separated) words defining the type of data, and 5 hyphens
an optional (and rare) rfc822-style header, terminated by an empty line
base64 of the data, broken into lines of 64 characters (except the last); some programs instead use the (slightly newer) MIME limit of 76 characters
a line like the BEGIN line but with END instead
Some readers check/enforce the line length and END line and some don't, so if you get those wrong you may create files that sometimes work and sometimes don't, which is annoying to debug.
Thus for example a PKCS1 private key (unencrypted) in PEM looks like:
-----BEGIN RSA PRIVATE KEY-----
MIICXAIBAAKBgQCjcGqTkOq0CR3rTx0ZSQSIdTrDrFAYl29611xN8aVgMQIWtDB/
lD0W5TpKPuU9iaiG/sSn/VYt6EzN7Sr332jj7cyl2WrrHI6ujRswNy4HojMuqtfa
b5FFDpRmCuvl35fge18OvoQTJELhhJ1EvJ5KUeZiuJ3u3YyMnxxXzLuKbQIDAQAB
AoGAPrNDz7TKtaLBvaIuMaMXgBopHyQd3jFKbT/tg2Fu5kYm3PrnmCoQfZYXFKCo
ZUFIS/G1FBVWWGpD/MQ9tbYZkKpwuH+t2rGndMnLXiTC296/s9uix7gsjnT4Naci
5N6EN9pVUBwQmGrYUTHFc58ThtelSiPARX7LSU2ibtJSv8ECQQDWBRrrAYmbCUN7
ra0DFT6SppaDtvvuKtb+mUeKbg0B8U4y4wCIK5GH8EyQSwUWcXnNBO05rlUPbifs
DLv/u82lAkEAw39sTJ0KmJJyaChqvqAJ8guulKlgucQJ0Et9ppZyet9iVwNKX/aW
9UlwGBMQdafQ36nd1QMEA8AbAw4D+hw/KQJBANJbHDUGQtk2hrSmZNoV5HXB9Uiq
7v4N71k5ER8XwgM5yVGs2tX8dMM3RhnBEtQXXs9LW1uJZSOQcv7JGXNnhN0CQBZe
nzrJAWxh3XtznHtBfsHWelyCYRIAj4rpCHCmaGUM6IjCVKFUawOYKp5mmAyObkUZ
f8ue87emJLEdynC1CLkCQHduNjP1hemAGWrd6v8BHhE3kKtcK6KHsPvJR5dOfzbd
HAqVePERhISfN6cwZt5p8B3/JUwSR8el66DF7Jm57BM=
-----END RSA PRIVATE KEY-----
The same key in PKCS8 unencrypted:
-----BEGIN PRIVATE KEY-----
MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKNwapOQ6rQJHetP
HRlJBIh1OsOsUBiXb3rXXE3xpWAxAha0MH+UPRblOko+5T2JqIb+xKf9Vi3oTM3t
KvffaOPtzKXZauscjq6NGzA3LgeiMy6q19pvkUUOlGYK6+Xfl+B7Xw6+hBMkQuGE
nUS8nkpR5mK4ne7djIyfHFfMu4ptAgMBAAECgYA+s0PPtMq1osG9oi4xoxeAGikf
JB3eMUptP+2DYW7mRibc+ueYKhB9lhcUoKhlQUhL8bUUFVZYakP8xD21thmQqnC4
f63asad0ycteJMLb3r+z26LHuCyOdPg1pyLk3oQ32lVQHBCYathRMcVznxOG16VK
I8BFfstJTaJu0lK/wQJBANYFGusBiZsJQ3utrQMVPpKmloO2++4q1v6ZR4puDQHx
TjLjAIgrkYfwTJBLBRZxec0E7TmuVQ9uJ+wMu/+7zaUCQQDDf2xMnQqYknJoKGq+
oAnyC66UqWC5xAnQS32mlnJ632JXA0pf9pb1SXAYExB1p9Dfqd3VAwQDwBsDDgP6
HD8pAkEA0lscNQZC2TaGtKZk2hXkdcH1SKru/g3vWTkRHxfCAznJUaza1fx0wzdG
GcES1Bdez0tbW4llI5By/skZc2eE3QJAFl6fOskBbGHde3Oce0F+wdZ6XIJhEgCP
iukIcKZoZQzoiMJUoVRrA5gqnmaYDI5uRRl/y57zt6YksR3KcLUIuQJAd242M/WF
6YAZat3q/wEeETeQq1wrooew+8lHl05/Nt0cCpV48RGEhJ83pzBm3mnwHf8lTBJH
x6XroMXsmbnsEw==
-----END PRIVATE KEY-----
and PKCS8 encrypted:
-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIC3TBXBgkqhkiG9w0BBQ0wSjApBgkqhkiG9w0BBQwwHAQIkErtXjGCalMCAggA
MAwGCCqGSIb3DQIJBQAwHQYJYIZIAWUDBAEqBBApOUG3MKrBC/5bDBH/s5VfBIIC
gN5o1aJxvJYbp2oA/quz+BnCFn8ts3wPPOcqarHddy0L/VH3BdqFNbnPZEaDnvDl
kqChRsti4AAeX18ItMeAyNLNFv0J4mfI8Q5E7iEnPp+dTsZqNfVIJe2NGxOS7zp2
oQQIZVgjW0akDehv6ZDN796qDBlMY2g80wbBrzVgMJu/byG9IQQjngUE9QNGwrsj
7lYSprxjfTZOk1aGBD0d/HsmetIJvCeJ2i/5xAiGgZRrSWMC6aN7Zlra3eIvHQTB
aKZ8/0IT3iKSr6FpkEopOQae8biiTEVGw9D339P3qOSs2ChWWF+OV2sEA67w6q5j
pz6Poc5jgq4FOcf06GdcVa4tst2uykNJCW0wHpcUR1Tr9ILLhrZPYBYVQWW53Eee
o4+mqW2YORdG3a/jLHpEjL0Vdg95QNpdZoMv8plotN1MUBLebd05aCe5hJUb/x74
3GTwmRGmKoHOhOO3hhUaMCmZIg1xPlNT3jqxrZDoATBeONbaFP8OOkeucVYHbdUO
Ad7z6J8XuZDoxM0BVrGykEiQL2nAOccdsGoC33C9hjkqgU8G9jWElbynJlVqv+1a
lFHWjX5lB6ueiY/rClpVlLmXGB83OVPlo70FV0B9rhRChyB1IJJRYPFDJHSHJNu+
Pqay8zw82Gh/G+TWH/JCLm5YjX55ZSFMUmvwOLaxyQpmAGNH6dIBTAaSctVA7UrV
jm7m+5T7seiNYNEi19vDJipgr0GbX8+np47VrsJDxsS20wTeA/9ltD0xXwNrEKHd
2Nv/1OaCgnBQHIGULgEn9pT3/vU87bBHYjVdrWoUmqd9zFYtdImQE9u8IKTxTe4R
UPRGHqltz4uOjbD1epkSGe0=
-----END ENCRYPTED PRIVATE KEY-----
Observe the type of data in each file (or other data unit) is easily recognized from the BEGIN/END lines. The actual key values in the data are not easily read without tools, although only the third actually needs secret information (the password used to encrypt).
PKCS#1 and PKCS#8 (Public-Key Cryptography Standard) are standards that govern the use of particular cryptographic primitives, padding, etc. Both define file formats that are used to store keys, certificates, and other relevant information.
PEM (Privacy-Enhanced Mail) and DER (Distinguished Encoding Rules) are a little bit more interesting. DER is the ASN.1 encoding for keys and certificates etc., which you'll be able to Google plenty about. Private keys and certificates are encoded using DER and can be saved directly like this. However, these files are binary and can't be copied and pasted easily, so many (if not most?) implementations accept PEM encoded files also. PEM is basically base64 encoded DER: we add a header, optional meta-data, and the base64 encoded DER data and we have a PEM file.
Given an arbitrary (valid!) private or public key encoded inside of a PEM, with the pre-encapsulation boundary and post-encapsulation boundaries intact, is it possible to know exactly what format the bytes take (i.e. are they OpenSSL traditional, PKCS8, X.509 SubjectPublicKeyInfo, etc.), or does one need some a priori information to properly decode them?
With certificates the situation is almost straightforward - there the boundary line specifies what is expected (a certificate or a private key).
In OpenPGP armored data the boundary line also tells you what is expected - the key(s) or the data.
SSH keys created by several SSH applications have the same boundary lines but different format of the key itself. So you need to try reading the data in all expected formats.
RSA public keys usually have RSA 1.5 format so you can assume that you have an RSA key.
PKCS#12 is not usually wrapped to PEM (I never saw such files). The same goes for PKCS8.
PKCS#7 certificate storages are sometimes PEM-encoded and they have something like BEGIN CERTIFICATE STORAGE in their boundary line.
To sum it up - to some extent you can rely on the boundary line text, but this doens't give you a 100% guarantee.
You should have a look at the specification, PKCS#8 is specified in RFC5958 in Section 5
When .p8 files are PEM encoded they use the .pem file extension.
PEM encoding is either the Base64 encoding of the DER-encoded
EncryptedPrivateKeyInfo sandwiched between:
-----BEGIN ENCRYPTED PRIVATE KEY-----
-----END ENCRYPTED PRIVATE KEY-----
or the Base64 encoding, see Section 4 of [RFC4648], of the DER-encoded PrivateKeyInfo sandwiched between:
-----BEGIN PRIVATE KEY-----
-----END PRIVATE KEY----
Slightly longer answer: "Between the tags is in any case valid Base64 Encoded DER encoded ASN.1"
How to convert X.509 certificate from a hex-dump form to .CER format? Additionally, should be blank space separators removed from hex dump first? Thanks.
You could use ASN.1 editor. It has a data converter that will convert HEX to PEM format of data. the source is also available so if you need to use it in code you can look how it is done.
Or you could use certutil.exe using command
certutil -decodehex c:\cert.hex c:\cert.cer
Normally .cer can either be binary encoded - DER - or it can be DER that has an "ASCII armor", called PEM.
If you want to create DER you probably only have to decode the hexadecimals. In that case you obviously need to disregard any whitespace within the hexadecimals (as well as any other spurious data that may be present).
If you want to have PEM it is required in addition to base 64 encode the result and add a header and footer text. Or you can use an existing library that does this of course.
I have a X509Certificate and I write/print it to a file as follows.
(I'm not writing encoded bytes, because I want to read the content of the certicate)
X509Certificate cer = generateCertificate(); // cer is DER encoded
writeToFile( cer.toString() ); // cer.toString() converts DER to UTF/ASCII???
Later I want to read this file (above) as String and create a new X509Certificate.
String cerStr = readCerFromFile(); // Read what is written above. In ASCII/ UTF format
ByteArrayInputStream bais = null;
try {
CertificateFactory cf = CertificateFactory.getInstance("X.509");
bais = new ByteArrayInputStream(cerStr.getBytes());
return (X509Certificate) cf.generateCertificate(bais);
} ...
This throws following Exception.
Java.security.cert.CertificateParsingException: Invalid DER-encoded certificate data
And it is obvious that I'm not converting cerStr to DER format (and I don't know whether it is possible to convert into DER ). Can any one please explain how can create an X509Certicate from a String which is not encoded.
Thanks in advance.!
The short answer: you cannot. DER encodes too many details that cannot be easily converted to and back from a String. You are better off simply saving the DER encoded certificate using cer.getEncoded() as GregS has explained in the comments.
If you want to see the the contents of the certificate, simply save it with a file extension that your operating system recognizes and double click it. If you want to have a command line method of printing the plain text information use e.g. openssl:
openssl x509 -text -noout -inform DER -in mycertificate.crt
Which is standard included or optional in many Unix flavours (Linux, Apple) and can be run on Windows as well.
When you take certificate as raw data cert.getEncoded() (in .Net it is cert.RawData) it is encoded in DER format. Informally speaking it is just a special binary representation of the certificate.
But there exists good string representation of certificate. You can convert raw representation of certificate in DER to Base64 formatted string. I don't know JAVA, so in .Net it will look like this Convert.ToBase64dString(cert.RawData).
You can save certificate in both formats to a file with .cer or .crt extension and open it using standart OS certificate viewer.
A poor mans answer (I would very much a lower level)
// create the pfx byte stream...
byte[] selfSigned = CertificateCreator.CreateSelfSignCertificatePfx(distinguishedName,
new DateTime(2013, 4, 1),
new DateTime(2013, 12, 31),
insecurePassword);
// crate a certificate instance
X509Certificate2 cert = new X509Certificate2(selfSigned, insecurePassword);
// export as .cer [DER]
selfSigned = cert.Export(X509ContentType.Cert);
// write to file..
System.IO.File.WriteAllBytes(certificateFilename+".cer"), selfSigned);
In Java you can do
String sCert = javax.xml.bind.DatatypeConverter.printBase64Binary(certificate.getEncoded());
In .Net
Convert.ToBase64String(Certificate.RawData);